Thank you everyone for making it a long first thing on a Friday morning.
I hear that you guys broke the dance floor last night, so yay.
Okay, so I'm Mia. I'm doing a PhD in digital humanities at the Open University in the UK.
My PhD investigates how academic and family and local historians evaluate, use and contribute to crowdsourced resources.
I've got a particular focus on how they manage issues around place and the geolocation of historical materials.
This follows on from earlier research on crowdsourcing games from museums and digital participation with audiences in museums.
This short paper is presenting some early findings for my work in progress.
To put this research in context, I wanted to focus on a particular group of people with similar research methods,
and being a technologist suddenly in a history department through the wonders of digital humanities funding programs.
I thought working on a group of people investigating the history of the same country in the same time period
would mean I'd have a group of participants who faced enough common issues with the availability
and types of primary sources from early modern England to provide meaningful analysis.
There's also relatively little work done on these kinds of specialist users in terms of public participation,
which isn't to say that there's no work done.
Some quick definitions for clarity. Crowdsourcing takes many forms,
and it's got that kind of buzzword thing where it's applied to many different things in quite extensive ways.
Here I'm interested in its application to resources not created by institutions,
or those co-created by the public within a framework set up by an institution.
The easiest way to understand this model of crowdsourcing is to think of Wikipedia.
It's an encyclopedia collaboratively created by members of the public rather than by invited experts.
A good definition for the specific kind of crowdsourcing that I'm interested in comes from the Transcribe Bentham Project,
which describes crowdsourcing as the harnessing of online activity to aid in large-scale projects that require human cognition.
Human cognition is a special stereo which is things that computers can't do very well,
visual processing, that people find really easy.
I'm particularly interested in making more of the activities that historians are already undertaking for their own purposes.
For example, lots of different people go into an archive, record information about a particular document,
they transcribe sections from it.
Is there a way that you could get all these different actions to be shared on a site so that they wouldn't be done again and again?
It's a waste of people's time to be doing this low-level grunt work,
although it's not to take away from the close reading that people are doing as well.
I'm interested in discovering under what conditions academics and historians would share the results of that work
with the projects that collected these snippets, so something like participant digitisation.
There's a phrase that's necessary preconditions for sharing with peers, and I'm interested in what they might be.
In general, crowdsourcing is a misleading term.
Most heritage crowdsourcing projects are closer to the traditional model of volunteering in libraries, archives and museums.
For that reason, it's sometimes called community sourcing.
In general, crowdsourcing projects divide up big, daunting tasks like digitising an archive into smaller, more manageable tasks,
like transcribing a name, a sentence or a page, or correcting an OCR error in a newspaper.
This method has been used to digitise vast numbers of primary sources.
But if crowdsourcing solves so many problems, then how come we're not all doing it?
So Hal's quote at the bottom raises one of the most important issues for historians and humanists.
Crowdsource projects are generally open to participants who are judged on the quality of their work, not their qualifications.
But how do historians know if those qualities are the same qualities that are important to them?
I was also interested in the impact of anonymity or pseudonymity, which is defined here as false names chosen by users of websites.
They might be stable over time, or they might be a name that you choose once a week depending on your mood.
There are many reasons why an academic might choose to disguise their identity, including masking their research activities from competitors and peers,
and a desire to conduct early exploratory work in private.
But if contributors aren't using their real names, they can't derive any authority from their personal or institutional identity.
How does this complicate the process of assessing the provenance of information on sites
when those sites include contributions from any random person with access to the internet?
So I decided to focus on academic and amateur or family historians,
because I thought the differences between them would be productive for understanding the impact of digitality on scholarly practice.
What is it that academics do only because of the pressures of academia, and what is common to most scholars?
But I also discovered that the definitions of amateur and academic aren't clear-cut.
My participants included people who trained as historians, but used those skills in non-academic contexts,
academics who learned their historical research skills in the job, and self-taught researchers with decades of experience.
Historians are perfect examples of pro-ams, these knowledgeable, educated, committed, and networked amateurs who work to professional standards.
So why study crowdsourcing? It's seen as a solution to the digitization backlog, but are these resources created actually usable by historians?
And how do these resources disrupt current notions of authority, reliability, trust, and authorship?
So just a note on methods.
I recruited participants through personal networks and social media, conducted interviews in person or in Skype.
So far I've had 15 participants, interviews last for about two hours, up to two hours,
and might generate up to 8,000 words of text to transcribe and analyze.
So my aim in this round of interviews was to understand current research practices and probe more deeply into how they assess different types of resources,
both traditional and digital resources.
To understand their feelings about resources created by crowdsourcing or by members of the public,
their use of mapping place names and geospatial technologies,
and to discover when and how they would share research data and findings.
So some early results.
Participants often cited their knowledge and experience as how they sniffed out unreliable documents or statements.
So they just sort of say, well, I just know, I compare it to my knowledge of what's normal in the field.
For secondary sources, variations on the way it's written were common.
So it's another example of the tacit knowledge that people bring to these tasks.
They cite things like the presence and quality of footnotes, the depth and accuracy of information.
But judgments like the quality of footnotes come through experience.
So I'm quite interested in how we can sort of unpack this tacit knowledge and help make that available to people who are just starting out
and help them make better judgments about the quality of materials.
So transcribed sources, text versions of original source documents, introduced another layer of quality assessment.
Researchers might check for transcription errors that are often copied from one database from another to verify a resource.
So if there's a database of names or place names and they know that way, way back in time one name was written incorrectly,
they'll look for that name to see whether this database is actually new or whether it's just repurposing existing material.
Most researchers used multiple sources to verify and document facts.
Everyone I talked to claimed to have perfect research methods and to take everything very seriously.
No one ever admitted to being a bad historian.
Although I found a few people who were kind of at very early stages of family history research who admitted that they don't actually care so much about the sources.
But anyone with sort of five years experience over claims to always check their sources and rely on multiple sources before regarding something as a true fact.
They also used attributes such as place names, occupation, age to disambiguate person names.
So that's to make sure that the person that they're looking for is actually the person they're interested in and to help narrow their searches.
It appears that there are several key stages between people accessing original records and publishing information.
So several key points where research data or findings might be shared.
They include the time of acquiring and transcribing records, producing visualizations like family trees and maps, publishing informal notes and publishing interpreted content.
The stages the researcher passes through depends on their motivation for doing their research and on their audience.
Information might change format between stages.
So it might go from being a notebook to a sort of a type written document to being in a database to being in a publication.
Some information would drop out at each stage because they'd sort of claim not to share information that hadn't been verified.
So there's a sort of funneling process in terms of what they'd share.
It seems in general that the size of the potential audience increases and the level of trust required to share with their audience decreases in later research stages,
which kind of makes sense. You might share raw material with a very close trusted colleague, but you'd let anyone read the final article.
For academics, there might be an additional post-publication stage.
When the resources are regarded as depleted, so they might have a personal database of work in the archives.
If they've published everything they possibly can from that database, at that point they're happy to share things.
Family historians see some value in sharing versions of family trees online or in posting the names of people that they're researching.
They regard it as a form of advertising that helps them find other people looking for the same names.
Sharing is often negotiated through private channels and personal relationships.
So methods of sharing vary. They include showing people work in progress on a screen rather than sending it to them,
using email in preference to using sharing functionality supplied by websites.
This is really targeted, localized sharing that allows the researchers to retain a sense of control over early stage data.
So this is one key area where identity does matter. Who you are and who you're sharing with is key at this point.
Information is often shared progressively, and getting access to more information depends on your behavior after the initial exchange.
So if you credit the provider in any further use of the data or if you reciprocate with data of your own, people are more likely to share with you again.
Some causes of resistance in sharing data.
For historians who are considering sharing their depleted personal archives,
being able to convey the context of creation and the qualities of the source material is really important.
If they can't do this, then they won't share.
And this is one area where identity is important because if you know that someone is a historian and you know their body of work,
you've got a sense of the kinds of materials they use and how they would have been treated.
Conveying information about data reliability is also important.
So some reliability information is implicit in the format as it relates to stages of research.
So comments in a notebook are just things scribbled down in the archives.
Things they've actually typed up are things that they regard as being reliable enough to invest that time.
They'll also use hedging phrases in text.
And people will kind of make snap judgments based on the number of corroborating sources
or a value judgment about the quality of the sources.
If it's difficult to convey levels of certainty about reliability when sharing data, it's less likely that people will share it.
Participants felt a sense of responsibility about not publishing, even informally, information that hadn't been fully verified.
This is particularly strong in academics.
Some people confessed to sneaking photos of archival documents that they didn't have time to transcribe in full.
If they're not meant to be taking photos in the archive, they're not going to share those photos because it might come back and bite them.
Researchers also didn't always see a point in sharing data.
Family history content is seen as being too specific and too personal to have value for others.
Academics may realize the value of their data within their own tightly defined circles,
but they might not realize that those records have information for other types of researchers.
And there are some concerns that are particular to academic historians.
Reputational risk. One researcher, the full quote, said,
We are wary of others trawling through our research, looking for errors and inconsistencies.
Obviously, we're trying to get things right, but if we've made mistakes, we don't want to have them used against us.
In some ways, the less you make available, the better.
So one consequence of tying identity to the information that's shared is,
if you've got that nitpicky colleague who's always looking for a mistake that they think can bring you down,
you're going to be very careful about what you put out there.
Scholarly territoriality can be an issue.
If there's someone else working on the same resources, their attitude will affect how much others share with them and in general.
If historians don't feel like they'd get information of equal value back, they're less likely to share.
And there's a sense that if people have done the hard yards and put the work in,
that someone else shouldn't be able to just come in and sweep all that stuff up.
Professionals also don't want to give away intellectual property.
Sharing data online is risky because, certainly for historians,
the protocols of citation and fair use are variable or unknown.
And another issue is, if you've contributed work under a long-standing pseudonym,
how would you be credited for that work?
In terms of people not using crowdsource resources,
the main cause of underuse is actually not providing access to the original documents.
It has nothing to do with identity, nothing to do with who actually transcribed or worked on that document.
People will use almost any information as pointers or leads to further sources,
but they won't publish findings based on that data unless the original documents are available or they've been peer-reviewed.
Checking the transcriptions against the original is seen as good practice
and part of a sense of responsibility to the world's knowledge.
This work is still ongoing, but so far it's encouraging to see that amateur and academic historians
share many of the same concerns for authority, reliability and trust.
So overall, the identity of the author is less important than you might expect.
When sharing digitised versions of primary sources,
reliability is not invested in the identity of the digitizer, but in the source itself.
Content found on online sites is tested against a finely tuned set of ideas
about the normal range of documents rather than the authority of the poster.
So, at least for the crowdsourced digitisation of crowdsourced resources,
it turns out that it doesn't matter if nobody knows that you're a historian.
Thank you.