Good afternoon. Hello. Hello. Yeah, we're good. This is a very strange headset. Well, good
afternoon. Thank you, Tonya, for the introduction. I feel as though in a room full of academics
that everyone may have had the experience when they were doing their graduate work of
friends or family looking at them and say, what exactly are you doing? And after the
introduction, and usually when I get introduced as studying in both English and systems design
engineering, I get that face, and this was, I think, no exception. There was a few that
perked up and said, what's going on? So hopefully, I'll be able to explain it as we go. I'll start
my presentation today with a brief introduction of my overall projects and then get into specifics
about the current visualization techniques that we are working on. And I say we because, although
the idea to start this was mine, there was, this is an interdisciplinary project that we've been
working on between the English department at the University of Waterloo and the systems design
engineering labs that I work in. In general, the project as a whole is twofold. In the
interdisciplinary nature, what I do is based a little bit on serendipity and a lot of me simply
being precocious. The engineers and computer scientists that I work with on a daily basis call
me the door knocker. I tend to wander around the math and engineering building, which is this
mystical place for an English student, and knock on doors and ask questions. And I've found a lot
of really interesting people to talk to and some unsuspecting mathematicians that look at me like
I'm absolutely crazy, which is okay, I don't mind. I've somehow convinced the good people at the
University of Waterloo that the only way to pursue this line of investigation is to stand in the
middle and look both ways. And our university in Canada tends to be quite, quite technological,
and they've somehow believed me. So this is sort of the product of that. So for me, my main area
of literary stuttery study is modernist poetry. And besides the excellent work of some of the
people here at this conference that have increased the size and speed of databases and
with text and searchability of text, which has greatly sped up my work, I'm not sure the
technology has changed the way that literary critics view literature. The critical process
seems to me to be untouched by technology. And if the first part of this process is interested in
why that is, then the second part is interested in how, and specifically what happens when existing
theories of literature and are put into negotiation with technology. So I guess one of the questions
is, can we view literature different through this technology? And this is what I'll talk about today.
The University of Waterloo, I'm fortunate that our English department operates the critical media
lab, whose mandate is to use English theory to create objects to think with. And these are
essentially half theoretical, half art projects that tend to engage the community and other scholars
in discussion about what we've been talking about. My project approaches literary theory slightly
differently because the objects that I create are not intended to spawn inspiration in regards to
the literary theories themselves, but operate under belief that technology can function as the
praxis to humanist conceptions of the world. I have to start by saying that this project is supported
by the critical media lab, the UW Touch Lab, and then the Social Sciences and Humanities Research
Council of Canada. So the visual technique that I want to talk about today came about from a lot of
this so-called door knocking. I was always drawn to poetics because I love the synchronic aspect of
poetry. And I've always thought of words in terms of space. After a recent talk I gave at a digital
art opening for a piece that I did, I had four separate people and there was really four of them
come up to me and said, you know what I think you might be, you might have synesthesia, which I
didn't even know what that was at the time. It's like when people hear colours or see different
things, but it's always very disconcerting to have an art project that you've done be diagnosed by
the people in the crowds. So this is sort of what constantly happens to me all of the time. But
always thinking about words in space, I was looking for a way to make 3D models of poetry,
and this is what you're looking at right now. And it's void of essentially explanation to start for
a very specific purpose. I want to take you through the process of thinking that got me here first
before I break down the actual process of developing these 3D graphs. And this is what led
me to the touch lab and engineering department at the University of Waterloo. The area of interest
is information visualization and human computer interaction. I brought them an idea about how to
map words onto 3D space, and they helped me with the details based on their mathematical expertise.
So I've done quite a lot of work on this poem. What you're actually looking at, so I'll give you a
little bit of a hint, is Blake's The Tiger. And I've done a lot of work on this poem. And when
dealing with poetry, there's always a danger that the exegetical techniques can become rote and that
the approach can become systematic. And I was looking for a fresh way to look at words. The
first question that everyone asks when they see this graph is what does it mean, which is the
appropriate question. A lot of times when I showed this to people in general after we created them,
I wanted to know what they were thinking just from the aesthetic visualization. And there's a lot
of different types of groupings and graphs, and it looks like kind of a spiderweb. But if you look
down sort of at the bottom here, I'm going to throw this mic somewhere. I mean, there's a lot of
different lines that are coming into certain places. And for me, the question that I kept getting was,
what does this mean? What do these clusters mean? The actual visualization program, there's a rotation
and a zoom function where you can sort of go inside the actual graph and take a look so you can
investigate what's happening. So I mean, if this was to be thought of as a practice to the theory
I was trying to engage with, I was interested in exploring the defamiliarization that was the goal
of early avant-garde artists. This brought me into direct conflict with the data visualization
techniques employed by these experts in the touch lab. Because to accomplish this defamiliarization,
I had to devise a way to display structures and relations without including the words in the
graph. So there has been many, many fights between me and computer scientists and data visualization
specialists saying you have to label the graph, it has to have more information in it. And one of
the things that is sort of in my mind, and from dealing with these guys, that the lacks of these
sort of labels tend to make the visualizations not as effective as they can be. But the idea is to
engage these on a level of aestheticism or on art so that you can ask questions about the poetry
without actually knowing what the words are that you're asking questions about. It's about
separating this out. The thought process for me on this was based on problems of reflexivity
on meta-language analysis. I was concerned that like abstract expressionist art, many data
visualizations need heavy meta-analysis to understand what they were displaying. Instead
of trying to avoid this problem, I wanted to push it all the way and engage these objects as an
aesthetic phenomenon. See if we could gain insights about them without knowing or caring what the
words were until the questions had been formulated. So this process led to some interesting
inquiries, but based on this mapping, which I'll outline in a few moments, the main question for me
or that I was getting were what are these clusters? And this led me to the question of Denis Diderot.
There's a danger in a project like this if it tries to incorporate existing theory and specifically
18th century enlightenment thought that it was stuffing a square peg in a round hole. And I know
that this is a fear. I was aware of this when engaging with this theory, but I think the
technology and information visualization in particular, if conceived correctly, can be the
practice for this type of thought. So with that in mind, I'll quickly outline the thought process
that had me so engaged with Diderot and his enthusiasm. And for me, the clearest conception
of this project that I've ever read is from James Doolittle's wonderful little essay,
Criticism as Creation in the Work of Diderot. And this is an example of me recognizing that
I can't summarize it as well as he can. So a lot of these sort of breaking down and specific points
came from him. So I engage with Diderot as an example of how 18th century ideas of art are
still relevant and in terms of technology can be engaged or expressed through computers and
specifically through information visualization. Diderot is the perfect test case for this line
of thinking because his system of understanding art allows a type of interaction and creation on
the part of the critic that I feel is possible and visualization is used as critical tools.
So for Diderot, the key to understanding this conception of art is based on his understanding
of the word nature. And I mean, this is mostly directly from Diderot and he says that nature
is everything and everything is either matter or results from combinations of material elements.
Any element has the attributes of energy and sensitivity, which is to say that all matter
acts and reacts. An intelligent being, which is us, perceiving action and reaction,
supposes them to be cause and effect and calls the supposed bond between them a relationship.
Where relationships are thus created, the notions of structure, form and function are also created.
And the accuracy of these suppositions is subject to confirmation by experiment. And this is what
really strikes me about Diderot that he's talking about having theories of art, but being able to
experiment with them. And this is what this project was for me in creating these types of graphs.
So trying to incorporate these ideas into technology means for me that visualization
itself can be that experiment. And for Diderot, its imagination, which must
predominate in the psychology and activity of genius, and is primarily in reference to this
imaginative activity that man is said to create. Human creative activity is this one with the
acquisition of knowledge. It should not in fact be called creative, but re-creative,
for it deals with already created things. And the process of creation is threefold.
The organism of genius is penetrated by the energy inherent in a particular phenomenon.
This energy is transformed and intensified into enthusiasm,
the force which drives the creative mechanism. And the first step in processing is thus the
arousing of this enthusiasm by the perception of the meaning of a particular phenomenon.
So next imagination is energized by this enthusiasm to form a quantity of images
by which that meaning is to be expressed. Now this for Diderot is the most important moment
of the process, the moment of inspiration. And finally reason brings order among the
images and gives them comprehensible physical realization. So this is a whole lot of theory
that really is trying to just break down this three-part example of how I think data visualization
can take these types of theories of art and express them through the technology. We can
see how this process can be expressed in terms of visualization. Step one is arousing enthusiasm
by the perception of meaning. And here this has been reinforced to me by the defamiliarizing
nature of the visualization and the questions that are immediately asked about the image. So this
is the experience of the image for Diderot. And second stage is the developing or grouping of
possible explanations. The third stage being reason which gives them as he calls it comprehensible
physical realization. So this is how we are this what we are trying to do first is come up with the
actual process which I'm going to get into right now on how to make these visualizations. The idea
is to take it from just a data visualization or maybe a semi-information visualization the
difference being in at least in our lab we consider information visualizations to be interactive.
That in this one on the left there is a zoom function and a rotate function where you can
sort of take a look at it from different angles. But the idea is to break this down
and have it so that as you click on the the dots that you see on the graph which actually stand
for words the words are going to pop up on this side in a window. And I was hoping to have that
ready today to show working but if any of you have worked with computer scientists they break
everything in terms of making things faster or better or you know more intense or something and
they just constantly break what I give them. So it's not actually in any working shape right now.
So what we're trying to do is we're trying to take this and make this into this type of interaction
where you have a 3D graph of a poem that you're looking at on the left side you can investigate
in the way that I'm talking about and then it will actually show you what you need on the right side
as you come along. So if we imagine this entire process being taken up with an information
visualization then the action-reaction is the de-familiarizing that I talked of earlier.
It is the re-representing of structures of literature or whatever else you happen to be
visualizing and the interaction with the visualization that is the creative aspect
that the critic can engage in. And in this sense for Didera the artist outlines it as an amalgam
of the creator of the original literary work and the data visualizer which is often the critics
themselves meaning that it's possible to take part in both these instances using technology.
So the reason why I chose to engage with Didera and why I like it is because he thinks
that the literary critic can actually be part of the process of creation and I think that we can
use technology and data visualization specifically to give tools to be able to actually engage in
that process. So information visualization is the perfect test case for a theory that's
streamlined like this. One where we can use technology to investigate the steps involved
in this type of phenomenological experience. This was my long answer to why I left words off to
start the data visualization and why I'm trying to bring you along with me before I explain
the actual graph. I was trying to engage with Didera's view as critic as creator.
Now I believe that the objective of data and information visualization is to gain new insight
into existing data sets and in my case all of the literary critics that I work with
are they just cringe when I call Blake a data set but you need to actually think about it when
in that way and the level of insight possible in my opinion works on a continuum of complexity
within the visualization. I believe it to be very simple that the more complex the visualization is
while still maintaining a balance of its elements the more likely it is that a critic will gain
insight from it. In the case of this text visualization the method was always intended
to be a heuristic for me. A proof of concept if we were working in industry and many of the
visualizations I have come across although aesthetically pleasing were impossible for
me to understand and this is because a lot of the times data visualizations need metadata
to actually be able to engage with them and I don't want to pick on the buoyant tools guys
because I have used some of their stuff and I really like it but I included this after I
looked at my own abstract online when I got the email saying it was available. I was reading some
other ones and saw these buoyant tools and was wondering if in fact my abstract had done the same
thing and it was and I wrote the abstract so I look at this and I know exactly what I am talking
about but I am not sure without reading the abstract that a word cloud just in general as
a visualization gives too much to actually go on about knowing what you are looking at.
So the idea for me was that if I am looking at this can we put this up and know anything that is
going on and I have used some word clouds to try and look at novels and you tend to get things like
a lot of prepositions and you tend to get a lot of character names and if you haven't read the
novels and you don't know anything about those characters it almost means nothing. So this is
an instance where and I don't think this is the buoyant tools problem I think this is just a word
cloud problem in general where that there are very specific instances where they are useful which can
be in shorter texts like using them for poetry which I have done as well but as soon as you get
into something that you don't know about or you are talking about in terms of very large corpus
corpora you run into this problem of it starts to fall apart without meta analysis. So I was quite
pleased when I saw this beside my own abstract for this talk. Interestingly I think word clouds can
be very useful in studying poetry but there is clearly a point where the amount of words in the
original document tips the sensibility of the word cloud to a place of unintelligibility and
this often occurs with word clouds of entire novels. So again we are back to the graph and
this is where I am going to actually explain what we are doing here. What we have done is the box
that you are looking at is actually 3D space and it starts in the bottom corner here at 000 and in
the top corner at 111 and it essentially gives us and I will explain the proof where I got it from
but it gives us an infinite space in this box to be able to include every word in the English
language. So the points that you are looking at which have been blown up so you can see them
are a one to one correspondence between words in the poem in Blake's poem The Tiger and the space
in the square that we have mapped. Now what ends up happening is the lines that you see are a
visualization of the spaces in between the words as you rotate it you can there is a starting point
and you can actually add vectors to it to actually walk through the poem in 3D space if you are
interested. So the actual proof comes from George Kantor and it is very simple in its conception if
not in the actual proof and I have tried to actually locate a copy for this talk to actually
put up and say look here is the mathematical proof it is wonderful except it was written a long time
ago and it is almost impossible to track down even though every mathematician I talk to seems
to know it well so it is this very strange you know door knocking problem for me where I keep
knocking on doors and no one has the answer that I want but what it does is essentially his proof
says that on any given number line there is an equivalent point in n-dimensional space which means
that I mean George Kantor was the pioneer of set theory and pioneer of these counterintuitive ideas
of infinity and he says that between zero and one on a number line there is an infinite numbers
of actual numbers and that every point on that line has a one-to-one correspondence with
n-dimensional space so I asked the question what happens if we treat the alphabet as a base 26
number line what happens if we put the conception of words that we know on this number line and
then graph it in space and that is what the process to actually get these 3D graphs was about
so the idea is that you have to treat the language as a set of numbers so at the beginning
and it really this mapping that I'm using today and I'll explain it as I go forward
is a dictionary mapping so it runs from just from A to Z and what it does is you're actually
putting words as a base 26 number in between zero and one and as we line it up it gives us the
entire language on a number line so if we were to use the word digital Kantor's algorithm actually
breaks it down very simply in to x1 y1 z1 x2 y2 z2 all the way depending on how long your
your word is and this gives us three coordinates which I mean we right now we have to just remember
that we're considering the letters as base 26 numbers and this gives us three coordinates
the the advantage of doing this is that we have a one-to-one relationship between every word that
we can come up with in the English language and the space in that 3D cube so if you're looking at
the 3D cube from here if the bottom corner was zero zero zero you're essentially getting
as close to zero dot a zero dot a zero dot a as possible and it spreads out in all three
directions to the Z and so this is what we use first and it's called the dictionary mapping
or we called it the dictionary mapping and it's because we organize words in general in this way
we're comfortable with this this is how we do it and what it does is it essentially lines up
an a to z dictionary in three dimensions based on this these coordinates and plots points in there
so what you're looking at right now is a base 48 one because what I'm what we wanted to do
was actually encode the the textual information within the mapping which means that you're
actually putting location information of words on a page into the algorithm that that graphs
and what this does is as you can see there's punctuation and then using python with the the
slash n is an end line it encodes it into the actual into the actual mapping so that we can
put it into 3D space and what this means is anything that you can imagine we know it doesn't
have to be words it doesn't have to be letters it can be location information it can be any other
type of data it can be metadata can be encoded in this way and can be included in these in these
coordinates and plotted in n-dimensional space if you want to do really complex mathematics you can
make it as long as you want and as deep as you want into dimensionality so one of the options
that we found was a lot of times you can then convert this to base 10 so that it's a little
bit more understandable because we use base 10 numbers all the time
so again what we're looking at here is we're looking at a one-to-one reference between
the words in the poem and then graphed into this 3D space that we've created based on the mapping
what we think that you can do is not only engage in the critical aspect as Diderot was explaining
in terms of these but what we can do is we can use this type of mapping and any other mappings
we can come up with to do things like incorporate the textual information that we need as we go
and I think that this is this is a step for data visualization so again this is what we're getting
at if we're imagining what the end and gooey is going to look like this is what we're looking at
as we as we go through it we're going to click on certain things and it's going to show up with
different words the one thing that i want to say before i finish is that that this is actually
scalable to large corpora so eric miller in his article is literature self-referential writes
that a simultaneous possession by the reader of all the words and images of middle marsh um ala
rasheh attempts to do and ulysses may be posited as an ideal but such an ideal manifestly cannot
be realized is impossible to hold so many details in the mind at once well i graphed all three of
these books in this cube so i again i don't like picking on people but to me this is a sort of an
answer to something in terms of switching how we think about words that can challenge those ideas
we have about literature and about literary criticism the idea is is that i mean the dots
are obviously much much smaller at this point but they're still in the same point and this is
all three all three novels graphed from a different angle in the cube so you can deal with large
large um corpora here one of the problems that you run into this is it just zoomed in what i'd like
to show because i'm going to show it on the next two graphs is if you can see it starts to actually
take the shape in 3d of the actual english language and how i know this is that i graph the
entire oed2 and then this one i'm not sure is showing up so well but this is the complete works
of shakespeare and they actually at some point when they're compared to each other as they get
too big there's this tipping point where they all look the same so there's some limitation to it in
terms of large corpses in that we get to the point where where it just doesn't mean anything anymore
these are again scalable searchable you can look into them and there's functionality where you can
click on each dot and it'll tell you what word you're looking at which is sort of an investigative
experimental tool tool and then this is the zoomed in corner of the blake poem and with this type of
mapping one of the interesting things that it does is it groups um like words together in space in
terms of their first three letters is what it it um it makes most important but it also shows you
with the lines going in and out these clusters of the poems actually addressing words more or less
so if you can see here these two that i've highlighted in yellow are and and what in the
poem and my first engagement it's sort of a literary criticism with this was that i never
really thought about these two words in terms of this poem as being spatially important and so this
this shift for me led me back to the poem to take a look at what these were actually doing
and it started to change my conception of how it was so i'm going to leave it at that if there's
any more questions i have um a program that's generally working if anyone wants to see it in
3d moving or if anyone has any other questions about what they can do i'd be happy to hear it
so thank you very much