We're talking about how the community can help the community in a certain process.
The environment we are working in is that you have a large collection of images, say
hundreds of thousands of images, and the community is interested in finding, searching that collection.
And we have provided a system which does search by facets, and you should realize the
problem in this system is with images, we don't have many metadata.
It's like pictures, the particular collection we are looking at is African American images
from the 18th and 19th and 20th century, and there are not many metadata associated with
each photograph or drawing or image.
And how do you navigate in a large collection like this?
So we have decided and have reported before on a faceted classification system.
And I will briefly talk about how that one works.
Once we have that established, we find out that there is a scaling issue.
If you have a large community working on that system and you have a large collection,
you're running into scaling issues.
And that's what I'm going to report today.
And we also will tell you a little bit about evaluation and user studies we have done.
So the classification system we are doing is a web-based application, and it allows
anybody who is interested in searching that to help find things.
If I'm finding a particular, I'm interested in mayors of the first decade in the 20th
century who were African Americans.
And I'm doing my search in the collection, and I'm finding certain things, and I can
now actually record the knowledge I've gained in finding these images by introducing a
facet into the existing system by putting them all into a new classification called
mayors in the 20th century, whatever my particular group of things is.
And then later on, somebody else who comes across that facet and might be interested
can use that particular search mechanism I have added to the collection.
So as the community of users searches and uses the system, they can add navigation means
and make it easier for subsequent users.
And as the problems becomes now, if you have more and more users and more and more items
in the collection, you are running into computation problems.
Here let me show you the system so that you get a feeling for what it is.
Here is on the right side you see the items of collection, brief icon, brief description
of what it is.
And on the left side you see the beginning, this here is a little bit reduced, here you
would see the rest of the facets which are used to search this particular collection.
So you have date, source, and other items you would be seeing on your full screen.
And users could log in if they want to add to the collection or can just search using
the facet search.
And we have now introduced because one of the problems we found out as users modify
the facets, there is a tendency of time lag that facets get approved by the community
so to speak as being useful over time.
And also disapproved only over time.
So if someone introduces asperious classification, which isn't very helpful to users, that will
stay around for a while and will mislead people.
So what we have decided to do is instead of letting everybody work on the one schema of
facets and make immediately changes to it, that they can have their own personal schema
which they can use for their own, and then there is an algorithm which will look at all
the personal schema and try to merge them into a global.
Basically saying any user can work with their own navigation scheme, but in order to get
into the global one, certain properties have to be true, namely that a lot of people, for
instance, have introduced some particular facet.
Then we put it into global.
So the global one is done as things coalesce and many users will introduce navigation helps,
then they move over into the global schema.
Now you can imagine if you have hundreds of thousands of users, that's extremely time-consuming
to compute, looking at all of those schema and trying to merge them into one over time.
And then we also added a history feature because everybody loves the way back time machine
that you can look at, and again, that is a very time-consuming thing.
So let me give you, for instance, here this is how you can classify a particular image
on the left side.
You would see facets and you can change the facets yourself, too.
And here, for instance, you see the personal schema and the global schema.
Here's the history.
You can go back in history as to when, what did the collection look like at a particular
time.
So we looked now, we have a high computation problem.
So we looked at the cloud and we started both the Windows Azure and Amazon and we decided
to go with Azure, not necessarily because it's better, but because our National Science
Foundation funded Azure.
So we have some real reason to go with it, too, but that was the main one.
The one thing I wanted to warn you is when you go into the cloud with the sophisticated
application, you are heading for trouble.
It took us a full year to move from the initial system, which was just on the server, running
on the server, to move it into the cloud.
And the key issue is, I wanted to highlight that, is any application which uses databases
just has problems, because we ended up having to go with Azure, cloud, Azure, SQL, and that
meant we have to put everything because SQL is not the same as what we used to have MySQL.
So we ran into problems there and porting the management system where we are using open
source Joomla was a problem.
So overall, the experiences here, you can just see how everything in the cloud is load
balanced and works extremely well.
Once we got it going, so we could support lots of users, but it was a very, very painful
experience to move it to the cloud.
Let me talk a little bit about the evaluation we did, so that you have some experience there
as well.
We have a large class where in social media on the web type of classes in the business
school, and we used the system and gave them tasks to, here's a classification scheme,
and that's the one we have given where a particular collection with certain facets, and we asked
them to go and do your own.
And so they developed their own personal ones, and you can compare the left facets with the
right facets, and they diverged quite a bit.
And so we ran then our global merging algorithm, and you can't read the details, but the merging
algorithm took those from various, put them together, and produced a new one.
And I think he's telling me, let me, for instance, in this one, you could see that
a personal one had introduced a new category called New York under position.
And we found position to be similar with location and merged in New York into the global one
location so that now location at Alabama, Virginia, and New York introduced a new category.
And in general, the students, we found the usefulness.
We did them after the fact tests where we gave them a search task, and they had at that
point only the global schema available.
And we tested them to see if there was a statistically significant difference in finding things
after they took the class than when before when they were just doing it without those
additional things.
And we found a significant difference in it.
So it seems to work.
But using the cloud as the one conclusion, I want to say, is painful.
Oops.