Sorry, this feels weird.
I feel like I'm a backup dancer for Madonna.
So I'll take that as affirmation.
I want to talk about visualizing this new and quite complex
dataset.
It's not strictly all that new.
It's actually been under development since the 60s,
but it was only finished a few years ago.
And it's a dataset which is quite hierarchically complex.
So I think it's something which is ripe for visualization
and is also really something where visualization is viable.
And I'll see whether or not you believe me.
So first, I'm just going to talk about the background,
the story of the data, and then I'll
talk about historical visualizations,
and then one final bit at the end, which I've changed.
So the data I'm using is the historical of the songs
of English.
It was published in 2009, finally,
as the historical of the songs of the Oxford English Dictionary.
And it's what we think is a very good dataset.
It was called the most significant tool
ever devised for blah, blah, blah.
So it's a good set of data.
I can't take any credit for it whatsoever.
I was not born when it was 30 years old.
And the way the thesaurus essentially worked
was this man, Michael Samuels, in the 1960s,
announced to the Philological Society
that his department was going to create a historical thesaurus,
a rearrangement of all the recorded words in English
semantically.
And he just announced this, and then afterwards told
the department this is what they were doing.
And one of my colleagues once said that back then,
professors had real power.
We could try and experiment where you all go back
to your own departments and announce things to them now,
but I don't know how much they're like it.
So anyway, what happened was they took all the recorded words
in English, that is, the contents of the OED
and other dictionaries, which did not include information that
was in the OED, and went through them all systematically,
taking off every sense of every word,
and taking every sense and writing it
on a separate piece of paper like this.
So you have something, this word established as the verb,
known to be in use from before 1325 to 1561
and archaic use in this state, with a definition
and a sort of initial sorting code at the top right.
And there ended up being 797,256 of these slips,
which then had to be somehow arranged,
and that's why it took 44 years.
And post-grad students like me did quite a lot of it.
So you were given drawers and drawers full of this.
You might be given about 1,000, 2,000,
and you just went through them and put them into piles.
I did parts of maths, and you literally
do just sit down and say, this works to do with the graph.
So I'm going to have a graph pile there,
and you do that for about seven years,
and you go slowly insane.
And of course, it was actually also typed up.
You couldn't, the slips were so thin and so paper-like
that you couldn't sneeze or open a window.
But luckily, we're in Glasgow, and you
don't need to open windows.
There was one summer in the 80s where apparently it was useful,
but other than that, it wasn't.
So eventually, what you end up with
is pages and pages of this, which
is just a printed version, but the database is just
as interesting.
So like I said, 797,000 words, if you like,
arranged into about 236,000 precise semantic categories.
So the data looks a bit like this when you
drill right the way down.
So these are all the words that I've ever
meant in the history of English, vocabulary.
And you can see them there with the dates of usage next to them.
And you'll see at the top, this is the heading for it.
So it exists within a hierarchical system.
We're in 0208040501, sits within 02080405,
within 0208040, and so on.
It's more complex than that, though,
because you can also have subcategories.
You can see that one, that's subcategory 10.
Subcategory 10 is very different from 0208040501.10.
There are two different types of subcategories.
Some are nested nodes.
Some are nodes which hang off an existing node.
And also, the status three-dimensional,
because you can have different hierarchical numbers
for different parts of speech.
So it's a very complex hierarchy.
And it's one where, if you're looking
to try to get statistics out of it, it's quite tricky.
So actually, visualization is quite useful.
So the way I chose to visualize it was by using a tree mapping
algorithm.
Some of you might have used this yourself.
There's a lot of very good ways of looking at your hard drive usage
using tree maps.
You may have seen them where it scans your hard drive,
and it says, there's an enormous great big file of photos
from your answer birthday in this folder.
Do you really need them?
And I get rid of them for you.
And it tells you where they are.
So to start off visualizing, you can take the state
and put it into a table.
You can then give the table some sort of contour
or some sort of, what I'm looking for, not context.
You can give it some sort of texture, really.
By coloring it.
So I've colored them here, black for old English,
words first sighted in old English,
and yellow for words that are first sighted in more recent times.
And then that gives you a very basic visualization
where you say, oh, look, most of the words for vocabulary
are quite recent.
Comparatively speaking, the data lasts about 1,100 years, roughly.
So if it's very yellow, then that's very recent.
And then you can do this over and over again.
You can take one step back.
It's not much for this.
But if you look at all of the words for words and phrases,
you get something like this.
And you see that all the words, meaning words and phrases,
are quite recent.
There's our original category at the top right.
You can see the two black bits that were the old English words.
And you take another step back and see this, 02804.
This is all the study of language and linguistics.
And you see it's quite light as well.
The only thing is there are some darker patches
where you can see the influence, particularly
of the Anglo-Saxon grammarians and of translations,
studies where they've developed words
to talk about linguistics and language.
There's the original category.
And by this stage, there's no point in putting the words in.
So you might as well get rid of them
and you end up with something at the Super Mario Brothers
level.
And at this stage, to point out that I'm no longer just
drawing tables and coloring them in.
This is an algorithm, which is a fairly standard algorithm.
R does it very well.
Jump does it excellently.
And at the University of Maryland at the HCI,
there are software that will actually do it pretty much
for you on the fly if you just feed it data.
So there's a reference to that in the abstract,
and also at the end of the talk.
OK, so that study of language, and it's fairly bright.
But if you take another little step back to all of language,
then you get an idea of a bigger sort of pattern going on.
What you have in language is this chunk here,
which is linguistics, surrounded by much older words,
which you can start to see areas of recent lexical growth
and non-recent growth.
So you've probably got the idea of the hierarchy by now.
So I can give you all of English as one visualization.
Oh, that's our original category right at the beginning.
And this is modern English.
And luckily, the screen's so big that I can pretty much
say that each pixel represents a word in modern English.
There's about, as far as we know,
469,000 words, which are relatively recently cited
enough for us to say these are modern or modern-ish.
And what you have here is quite a strong patchwork effect
of all of these words.
Now, you remember from the way that we moved out
of the hierarchy that if there's words in one category,
then they're placed near words which are semantically close.
So this is all arranged semantically.
If you take two points other than some of the boundaries,
you can be sure that this word here and this word here
have something similar in common.
They're about the same semantic field or semantic set.
The most striking thing about this
is that it represents, it has this really strong patchwork
effect.
And that represents lexical growth in English
at quite a substantial degree during a recent period.
And I'll show this by going back a bit.
Firstly, here's a key to show you what it actually means.
Otherwise, it's a bit pointless for this actual data.
So you can see, for example, that physics and chemistry,
this little chunk here, is much lighter than what surrounds it.
You have this nice section here, number,
which includes mathematics.
The really bright yellow bits there,
that's computing the manipulation of numbers,
whereas that little dark black knot in the middle of those
are the lexical numbers 1, 2, 3, 4, which are very old
and I was actually.
And you can see some other patterns going on.
Leisure is quite bright.
Communication has big chunks of white in it.
Whereas these sections here, action, space, time, movement,
existence, creation, are quite dark.
Those are the old bits of English,
the old words that are in there.
So I said that this patchwork effect was unusual.
And I can show you why I think it's unusual by taking a step
back.
This is one slice.
This is modern English.
If you're back to English in the age of Samuel Johnson,
any words which we have evidence for being cited
during the lifetime of Johnson, or where
there's evidence of citation consistently and Johnson's
lifespan intersects it.
Or intersects it, sorry.
You see this, and there's almost no hint
of this very heavy patchwork effect.
It's quite well mixed.
Go back again to the time of Shakespeare.
Can't see it very well on this projector,
but it's very well mixed at the time of Chaucer.
It's really mixed.
So what you have are two different sets of change
in the history of English.
One, which I'll take the top right,
which represents rapid growth in essentially new things
to talk about.
And then you have stuff here in Chaucer's time where really
there were new words, but they were spread around the place.
And the sort of linguistic trauma that happened there
is very different.
When I talked to a friend of mine
about the top right one, the modern English one,
I use an analogy of a blob of Bluetack,
that stuff used to stick up posters.
That usually blew.
And if you take a blob of that and you stretch it apart,
then the bits you hold will stay blue,
but the bits you stretch go lighter, like a looking of white.
And that's sort of like this, where
you see these patchwork effects.
It's because English has been stretched apart
to accommodate new ideas.
And I thought it was a really good analogy
until she said, ah, yes.
So this is like the stretch marks of English.
And I thought, and I really wanted to call the paper that.
And I chickened out at the last minute.
So she was a non-native speaker.
It's not deliberate.
But anyway, those, you can see there,
are the stretch marks of English.
Another nice side effect of this kind of data
is that you can also show the relative size of sighted
vocabulary in these four periods as well.
OK, so the color is just different to make
it clear in this slide.
Oh, that's the wrong one.
The color doesn't represent anything.
The most saturated color represents
the end of the period.
And black represents old English.
So for this one here, which is the lifespan of Chaucer,
then the most saturated blue represents really
Chaucer's date of death.
And the reason why I've chosen different colors
is just because when I put it up on this slide,
I can remember which one's which.
Because I'm really bad at these kind of things.
So that's what the colors mean, sorry.
So some other things that have been done with it
are, if you look at the section here which
has these divisions, as well as looking at such things
as the nature of the trauma, the nature
of the expansion of English at different periods,
I was teaching metaphors to some students, a couple of,
well, not last semester, but the semester before.
And we were talking about how metaphor really
involved the mapping of one thing onto another,
describing one thing in terms of another.
And if you do that with this, we mapped a bundle of metaphors.
And you can see ones like here where this is travel,
and that's emotion.
And that's armed hostility, and that's emotion.
And those two lines represent love is a journey,
and love is war, as two separate ones.
You have other ones, life is a journey,
travel to existence, food and drink to mental capacity.
So when you digest ideas, these kind of ideas.
So we map these ones just to give them
an idea of what's going on.
And you'll notice that there are thick gray lines dividing this
in half, then dividing it across there.
These are the three main divisions of the thesaurus
into the physical world, the mental world,
and the social world.
And the interesting thing is that almost all the metaphors
we could find always bisected one of these major lines
between the physical world, the mental world,
and the social world.
Now that's sort of to be expected.
That's a point of metaphor.
But it did lead to some interesting discussions.
And also we have a project at Glasgow now called
Mapping Metaphor, trying to look at this more statistically
and more accurately.
OK, so this was mostly what I was going to talk about.
And I was very fortunate to get some good feedback
and to get some really nice comments from, by nice,
I don't necessarily mean, oh, that's nice.
I mean, kind of detailed and systematic and academic
comments on my abstract for this talk.
And it led to a few different things
which I'd like to show you now.
This is category growth, which you
can't see terribly well with the light.
But essentially, these are categories.
And they are colored according to their size,
according to their actual size, if you compare it
to these kind of ones where if it's big,
it travels big because there's lots of words in it.
These squares are colored according to their size.
And they are colored according to how much growth
they experience in this period.
If they're red, then they experience negative growth.
And if they're green, then they experience positive growth.
And what you can see is that it's quite hard at times
to distinguish in the thesaurus the relationship
of one category to another just because
of the nature of the data.
But what you can see is when you start to visualize it,
you can see instantly where there are clusters of words
which experience growth and where there aren't.
So you have a few ones here that are just big categories.
So in the period 700 to 1800, there
was a growth in our knowledge of electromagnetic energy
and also in the management of money.
That turned out well.
But you also have a few clusters around.
You can't see the clusters terribly well,
but you can download the slides.
I'll tweet about them later on.
You can have a look at the actual images a lot better.
But you see here, there's three or four
I can see here, language families have grown.
And I believe that's because of the nature of what
we know about that.
Geology actually grew quite surprisingly well.
And journal and newspaper, an interesting thing
is you don't get journal or newspaper very high.
If you do a ranked list of the categories that have grown,
you don't really see anything do with journal and newspaper
quite high up.
And that's because all of the individual categories
increased just a bit.
New words for journals increased just a bit.
New words for newspapers increased just a bit.
New words for magazines increased just a bit.
But it's only when you see them all clustered together
in a little green knot that you realize that the whole area
experienced a sort of lexical growth.
And you can do it with negative category growth.
You really can't see this well at all here.
I apologize.
But like I say, if anyone would like the slides,
they can have them.
And there are a couple of areas where
you'll notice quite substantial decreases, clusters
of decreases in English.
And that's both in moral courage and of the artifacts of faith.
And if you want to make connections between the two,
then go ahead.
I'm not going to put that there.
And this is also one which represents the age of Shakespeare,
at least the lifespan of Shakespeare.
It's one of the recorded constant truisms
about the history of English that Shakespeare
coined lots of new words, or at the very least,
Shakespeare was a first sighted evidence for a lot of new words.
But actually, when you look at the growth and categories
in that period, you see that really during the age of Shakespeare
or during the age that we have work cited by Shakespeare,
everything sort of experienced just a little bit of growth.
Shakespeare didn't really innovate to the extent
of creating new categories.
I didn't innovate in any particular period.
By Shakespeare, I mean the Elizabethan authors.
And finally, and this was an idea which I do owe to one
of the anonymous reviewers, and I thank them for it,
this is a measure of salience in a corpus of writing
of Thomas Jefferson, which I was working on separately.
And what I've done here is I've taken the categories of,
awesome, I've taken the categories of Jefferson's,
sorry, I've taken a large corpus of Thomas Jefferson
and compared his word usage to a reference corpus
and got a measure of salience, a log likelihood
measure of salience.
I've then taken all those measures of salience,
attached them to a thesaurus category,
and then averaged out the measures of salience
to categories.
And this is sort of like a fingerprint
of the Jefferson corpus.
These are the areas where they're darker,
or areas where Jefferson's talked about something
statistically more often than the reference corpus would have.
And you get some nice patterns there.
If you know things about Jefferson,
he talks about greatness quite a lot.
He also talks about leisure.
Some of those really bright ones up there
are American football, which I wasn't aware
he was really into, or at least words
to do with American football.
And bizarrely, a couple of other ball sports.
Command and order, which he issued quite a bit of,
and greatness in quantity.
Jefferson was the president who initiated Louisiana Purchase.
So he talks a lot about expanding and becoming greater
and becoming bigger.
The bottom left hand area is something
which very speculatively, I've done this
with a few other corporal letters,
this area, which is roughly around about the area of courtesy
conduct, forms of address, addressing other people,
and on to other people in polite ways,
tends to be quite dark in any area or any corpus
where you have a lot of letters.
This is a sort of letter fingerprint,
because these terms come up very often.
You're sincerely, you're dearly, dearest,
and, of course, the usual phrases
which you get at the start of letters in the past,
or at the end, you know, I am, et cetera,
your most beloved servant, and so on.
And a few things that are missing that you might think
should be there, and that's one of the interesting things.
I was expecting a lot more to come up
with farming and farmland and land in general.
Jefferson was a big farmer.
He was very interested in the farming of Monticello,
mostly because, you know, he had no money.
Jefferson sort of screwed himself over the Louisiana purchase
by making land America quite cheap,
and that's not good if you're a landowner,
but, you know, nonetheless.
And the corpus is biased towards such things
as one of his major books, probably his only major book,
is Notes in the State of Virginia,
which is just an introduction to the state
to its flora and fauna.
So, interestingly, that doesn't come up.
You can see a couple of dark bits there.
That's farm animals, and this one there is woodland.
But otherwise, you don't see a lot of this.
I find that interesting.
I'm not really sure what to make of it.
Like I say, these last ones were just ideas
of how to take the visualizations a bit further
than the original mechanism I was using them for.
So that's mostly what I wanted to say.
I wanted to try to show you why this data.
I hope you got some idea of how complex the data is
and why it's necessary to find some way into it
that isn't just looking through ranked lists
and also needs some sort of good statistical measure
to do with the relatedness of categories.
I don't have that yet,
so it's why I think a visualization approach
is quite valid and quite valuable.
You can also see such things.
If I flick quickly towards this,
then this idea of the stretch marks of English
is quite a nice one, well, apart from the name,
mostly because for a lot of time,
when you teach this kind of stuff,
when you teach the recent history of English,
you find yourself very often repeating truisms
from the textbooks that you're not entirely sure
have a solid evidence base,
or at least I'm not entirely sure
they have a solid evidence base.
They're just based on observations
through a long period of people working with the data,
which is very valid, obviously,
but requires a bit more information.
The fact that English, in the span of English,
there were various different periods of growth
is always very well attested,
but the fact that these periods of growth
were very different in their nature,
and they were very different in their distribution
across the semantic space of English,
I think it's a valuable one.
I hope I've also showed some other ways
in which it could be used.
Like I say, I thank the reviewers
for giving me some pointers towards other ways
that could be done.
So thank you.
Thank you.