So, at 4.30 in the morning on September 4th, 2010, a 7.3 magnitude quake struck west of
Christchurch, New Zealand's second largest city at a depth of only 10 kilometres.
While New Zealand is known as the Shaky Isles, because of its long history of earthquakes,
it sits directly over the Pacific Ring of Fire. Christchurch was seen as the most unlikely
place for a major earthquake to hit. While geologists knew it could happen, and that
if it did there would be major issues with liquefaction where the ground turns to liquid.
The general population were almost completely unprepared for a quake of this magnitude.
The earthquake caused significant damage, as you can see in this image, but miraculously
there were no deaths. Experts attributed this to the fact that the quake hit in the early hours
in the morning when most people were in bed. Over the coming months, although shocked,
the people of Christchurch felt they'd dodged a bullet, the big one had hit, and they'd gotten
away scot-free. That was until February 22nd, the following year, at 12.51pm.
We were weird that people of that sector were meant to get people in looking for survivors.
We thought this year was going to be a real deal, just to let speakers know that it was a violent deal.
I forget the most amazing stories of survival, people wishing no injuries, and others were coming
to send their fifth injuries as if they were real.
I don't believe it. Why this building?
I don't believe it.
Just coming to check, coming to work, thinking, you know, see if they all like what's going on.
We're really quite stranded, really. Right at the bay, obviously.
I'm not sure which is the worst.
We're not really sure.
They brought the rolls again tonight, so we'll just think about that later.
So the earthquake that hit Christchurch on February 22nd, 2011 was actually of lower magnitude than the one that hit the previous year.
But for a variety of reasons related to plate tectonics and Christchurch's immediate geography, what they call the peak ground acceleration was one of the highest ever recorded.
People talk of hearing the pea wave or pressure wave several seconds before it hit and the noise being intense.
185 people died in the quake, mostly in collapsed buildings or from falling masonry, and 80% of the central business district was damaged beyond repair.
6,000 homes and entire suburbs have had to be abandoned, and it was the third costliest insurance event in history.
The earthquake has been followed by over 12,000 aftershocks, turning what could seem to be from the outside an isolated event into an ongoing disaster that would take years to subside and perhaps decades for the region to recover.
And what have we done, then, as digital humanists?
Our goal, inspired by CHNM's 9-11 archive, was just to build an archive or an archive system that could store the vast quantity of digital material being produced after the earthquake with a view to documenting not only the immediate aftermath, but the perhaps decades-long recovery process.
The first thing we did was pull together a consortium of 10 cultural heritage organisations, bonded by a memorandum of understanding, signed at chief executive level, promising to work together to gather, manage, share and make publicly accessible all earthquake-related digital content.
The consortium is led by the University of Canterbury Digital Humanities Program, and composed of Archives New Zealand, Christchurch City Libraries, the Canterbury Museum, the Canterbury Earthquake Recovery Authority, the Ministry for Culture and Heritage, the National Library New Zealand on screen, the Naitahu Research Centre, to PAPA, which is our national museum, and the New Zealand Film Archive.
We've also established relationships with large content providers, including Fairfax Media, the New Zealand Historic Places Trust, Environment Canterbury, the Rural Earthquake Commission and several others.
And this isn't to mention the research teams we're working with across the University of Canterbury, which at last count had over 180 research projects related to the quakes.
And what have we produced?
The first thing we did was establish a web presence at seismic.org.nz.
If you wonder, seismic originally stood for Canterbury Earthquakes, Images, Stories, Media, Integrated Collection, but we dropped the full mouthful just before launch.
We're working with an external vendor for seismic.org, a software development company called MV Interactive.
To kick things off, I led an information architecture workshop where technical teams from across the consortium meet to discuss the nature of the site and how we might integrate our collections into it.
And the decision was made to develop a web presence and then add a federated search function that could surface content from all of our archives.
That search service went live a couple of months ago and now surfaces about 9,000 objects.
We're aiming to collect far more than that, but development of that part of the project has slowed down while we get our university research node live.
The goal now is to keep adding to the content across the UC seismic ecosystem as a whole so the search results continue to grow in number.
And this is the architecture of seismic.org.nz federated search.
The site itself is basic. It's just a content management system with a search box on the front page.
The key to the federation is a metadata aggregation service provided by a unit of our national library called Digital New Zealand.
Digital New Zealand contains over 24 million metadata records of library and cultural heritage items.
When we first started designing the federation, the feeling was that our university node, which I'll talk about later, would provide the aggregation service.
But we opted instead to use Digital New Zealand and tailored our ontology to make the best use of it.
And the process is simple. Digital New Zealand have created a UC seismic collection within their publicly accessible API, which earthquake-related content is harvested into.
In UC seismic, the seismic.org search simply queries this API and returns the results.
And the great benefit of this approach is that we can work closely with the team at Digital New Zealand to manage new and existing content providers.
They already have terms of use with over 200 providers and are very keen to build the earthquake collection.
And in fact, what we found is that a lot of our consortium members mandated as national agencies to curate and collect and maintain earthquake-related digital content.
And you can see the great strength of the federation in this diagram.
It's largely based on a backbone of significant national archives, but we're also aggregating smaller community sites we consider to be valuable to.
And this means that we're creating a broad collection that we can be fairly sure will be maintained in perpetuity.
Because the national agencies are part of a broader national process of long-term preservation.
Digital New Zealand regularly check both large and small sites to ensure they're still operating.
Which will allow us to step in and support those that might be at risk of failing in time to come.
And I'll talk about UC Quakes studies next, but you can see where it fits in here.
Because we initially designed it with a view to providing metadata aggregation services ourselves, it has the capacity to develop a mini-federation within the broader UC Seismic Federation.
It also allows us to work with community sites who may not have very good metadata with a view to harvesting their content and improving their metadata before contributing it to the broader federation.
Seismic.org.nz is an example of a system created relatively easily by emphasizing the importance of information architecture early in the design process.
With the architecture established, the actual technical build was quite straightforward.
Our focus on the creation of a healthy and sustainable federated ecosystem, however, implied the need to analyze our broader national digital infrastructure and, if necessary, fill in the gaps.
And it didn't take long to realize that there was a significant gap in that infrastructure in the academic and research context.
New Zealand was well-served for cultural heritage archives, and a couple of additional sites have been built to deal with specific kinds of post-earthquake material, but we didn't have an archive designed to store research-oriented data.
And building what came to be called UC Quakes studies, which should be going live in a matter of weeks, has been the biggest job our project has faced.
Quakes studies is an enterprise-scale digital object repository built using Drupal 7 and Fedora Commons.
Its original purpose, and still its primary purpose, is to store research data generated from university researchers.
Having said that, it has become apparent that this content is going to be a small percentage of the entire collection.
It seems likely that it will be mainly used to fill a gap in our national digital infrastructure, storing and making publicly accessible content from local government agencies, community groups, NGOs and individuals.
And the Quakes studies architecture seemed quite straightforward to many of you, I think.
We started the build using Islandora, made a pretty big decision to change tack and build a Drupal 7 Fedora Commons solution from the ground up, using a Drupal Fedora REST API module.
I won't go into the details behind that decision, but suffice to say we're pretty happy with what we've got, and considering open-sourcing it for others to use.
So Quakes studies sits on university virtual infrastructure, backed up to tape as part of normal IT services.
We've had to progressively scale the solution as a design process is proceeded to cope with increased demand.
This project really started off as a server under the desk, by the way, and it scaled to become a university-wide enterprise server.
So as project manager, I've had a few challenges managing that scope.
Creep isn't really the right word.
So we've gone from one virtual server with two CPUs to production and UAT environments with four servers and 10 CPUs.
Our production environment has twin Apache and Tomcat servers for the Drupal front-end and Fedora Commons back-end, respectively.
Drupal just acts as a thin client, and the Drupal Fedora REST module allows the front and back-ends to communicate.
We use Sola for search, in combination with Fedora's built-in generic search module.
I think the really nice thing about Fedora Commons is that it includes an RDF triples resource index, which we expose through an API.
One of my first requirements when talking to the developers for Quake Studies is that I wanted the system to be RDF-capable,
and using Fedora Commons gave us that out of the box, effectively.
Identity management is provided through our university's LDAP service, and the UC Seismic team, we have a program office,
with a manager, also have FTP access to allow us to perform bulk ingest of hundreds of objects at a time from the back-end.
We've also got solid access directly into the back-end for our vendors and educational systems developer in Wellington,
and as an ingestion channel for content within the Quake Studies mini-federation I talked about earlier.
End-users can gain access through an authenticated API, or via a secure web front-end.
Given all our content is aggregated upstream, at Digital New Zealand as well, I think we're here to say we've built a platform with huge potential for open access and content sharing.
In a similar vein, because we're sitting on virtualised infrastructure and using Fedora Commons,
which can theoretically scale to hold 10 million objects, so I think we'd have a few performance issues if we got to that number of objects,
we're well positioned for future increases in demand.
The next scale-up, if we had to scale up a step, will probably be towards a clustered solution, which we'd like to avoid if at all possible.
I guess the key thing for this talk is not so much the technical aspects, but where the digital humanities fit in the post-disaster Christchurch situation,
because I think we fulfil quite a specific and important role in the broader landscape.
Firstly, our emphasis on open access and open source have made it clear to prospective partners that we're going to be good citizens.
It's made it a lot easier to gather a solid consortium together.
Looking back at the early set-up and development of governance, it astounds me that we've managed to develop a memorandum of understanding across 10 government agencies.
It's fairly weak in some senses.
Each of the agencies only need to give two weeks notice before they pull their content.
But at the same token, the principles of open access, sharing, community, collaboration are enshrined in that memorandum of understanding.
We've emphasised from the start honesty, trust and sharing.
It's strange because these values come straight from Twitter.
These are the digital humanities values that have enabled us to communicate well with a broad variety of stakeholders, present ourselves as honest and collaborative and pull together a good ecosystem.
We try hard to live up to those values.
We frequently get presented with challenges that might force us into a corner.
We end up just shrugging our shoulders and say, we're digital humanists.
We have to open the doors.
Everyone's welcome.
We're going to try and support everyone who wants to come along.
In fact, we've become quite radical about it and even said, even if you want to work outside our ecosystem, we'll use whatever resources we've got to enable you to do that as well.
We emphasise the UC seismic ecosystem over individuals or organisations.
This has allowed us to avoid existing power differentials and build a conversation focused on the task at hand.
We have program meetings.
We have local libraries as well as the national library.
We have local museums as well as national museums.
We have a federated archive that includes websites built by, in one case, a young entrepreneur.
Young guys just built a website as well as major cultural institutions.
It's the digital humanities values that were baked into the program that have allowed us to do that.
The fact that we're university researchers has really helped, I think, because it suggests we'll be independent and look to future generations.
When the editor of the press, the local Christchurch newspaper, announced that he was going to give us all their content,
this is PDFs of the newspapers, all their images published and unpublished, their videos,
which for three days of the main quakes, September, February and June, when there was another reasonable quake,
amounted to 170 gigabytes of content, he said,
we're happy to give you this content even though we're a commercial organization because you're a university,
you're researchers and we can trust you to look after our content in an appropriate way.
And ironically, the fact that it was open access was an even bigger draw card for them because it meant that we weren't going to lock it away and monetize it.
We were going to open it up and effectively create another publication channel for Fairfax Media's content.
Since then, that offer of content has expanded from simply being the Christchurch press content to being all of Fairfax Media content across all of their newspapers.
There are some ongoing issues, the conversation's continuing, but in principle, that's where we're heading.
Having said that, the fact that we're researchers has helped.
Our real success has come from, again, digital humanities values and coming out of the ivory towers and engaging with the community.
In the early conversations, there was quite a lot of resistance to supposedly imperialistic university.
A lot of people in the community made an assumption that the university would be imperialistic and we're trying to grab their data and use it for our own purposes.
There are a lot of issues across Canterbury with what they call in research fatigue.
One story is there was a PhD student went into a shop in Littleton, which is a port town, and said, could you please fill out the survey?
He said, I'd love to fill out the survey, but I've got another five that I've got to get to first.
Digital humanities values, I think, helped us break down those barriers as well, because we've been going in at the same level and saying, we're offering a service.
If we can help, then sure, give us your content, we'll work with you to improve the metadata, store it, and make sure that it's safeguarded in perpetuity.
But if you don't want to get involved, that's cool.
And finally, in all our design and architecture meetings, we've emphasized the human element.
Despite me showing you the technical diagrams, our project and our program isn't about computer systems as much as people.
And if we can offer anything as digital humanists, it's surely a reminder about that.