Okay, sorry for the delay due to technical problems and welcome back to statistical methods
of language technology.
Last lecture we talked about the engram model, a workhorse for modeling sequences of words
with probabilities.
And we have looked at various ways of backing up, various ways of smoothing because we wanted
to solve the problem that we have to assign probability mass for unseen events.
You remember we had an example where we had an unobserved trigram in a trigram model which
would result in the probability of zero for the sequence if we do maximum likelihood estimation
and from this a lot of mathematical formulations followed in order to alleviate this problem
somehow.
Then we have started with the neural language modeling which is actually one way of dealing
with this.
Neural language models in a way naturally smooth over the data and for this we have
shortly repeated or introduced artificial neurons.
For those who haven't heard about these yet, I'll try to introduce them on the level so
that you have a basic understanding about what would they do in language applications.
If you want to learn more about these, there are dedicated lectures for machine learning
for neural networks where you can learn all the details about these things.
What we're going to use in the following is two notions.
One is the notion of the artificial neuron itself.
It has some inputs.
On these inputs are weights and what usually has been done, you sum up the product of the
weights and the inputs and you send it through a non-linear function.
In this case it's the sigmoid function but it could be a different function to generate
a numerical output and you could either use this output as input in yet another layer
of neurons and you could have many many of these neurons in parallel or you could use
them for an output.
For example if you want to learn a classifier that tells cat pictures from dark pictures
you would want to tune these weights as to predict what it should predict to learn a
function.
What we also looked at last time already was the recurrence in neural networks.
Here we have a typical setup in the sense that you have an input layer, might be more
than one neuron but here's only one.
We have a hidden layer and we have an output layer.
Here is basically where it should produce cat or dog, zero or one in this sense.
It would be the picture and the hidden layer can be connected to itself via time-delayed
connections.
This means the output of this neuron is fed of course to the output neuron but it also
is fed to itself in the next time step.
If you want you can imagine this as being an unfolded fashion so you could copy this
thing a few times and unfold these recurrent links so that this is the situation at time
step t1, this is the situation at time step t2 and whatever is the output at t1 goes into
the copy of the neuron at t2 and of course you would have to copy this indefinitely for
all the time steps.
Then we want to use these notions to define neural language models.
The idea is still that we want to assign a probability of the next word given a bunch
of preceding words and we still want to use this to define the probability of an entire
sequence such as a sentence.
The idea of neural language models as pioneered by Benjio and others in 2003 was to basically
work on how we represent the words.
In the old angry model each word was a symbol, different words, different symbols, different
counts, different statistics, different probabilities.
Here the idea is basically that we map our vocabulary in our training data to n dimensional
vectors.
These vectors have in this notation m dimensions, they are real valued vectors so it could be
any number minus infinite to plus infinite if you want.
Every word is associated with one of these vectors and they are called low dimensional
vectors but it is not low dimensional in the sense that it is a low number.
We live in the three dimensional world so it is very hard to imagine something high
dimensional.
Here we are talking about 100 dimensions, 200 dimensions which seems a lot if you compare
it to the 3D world however it is much less than if you would for example use one dimension
for each word in the vocabulary.
This would be millions and now we are down to a few hundred so in this case it is low
dimensional.
So we basically want to use much less dimensions than words in the vocabulary and then what
we have to do is we have to express the probability of what is the next word given preceding words
in terms of these vectors.
So what we do is we want to have a function that takes the few words before our current
word at time step t and the word itself and this function should return a pseudo probability
kind of a conditional probability that given that we have observed stuff in the past what
is the probability of the next word and since it should be a probability of course if we
would substitute all possible words in here all possible words in the vocabulary as a
next word then these pseudo probabilities would sum up to one otherwise it is not a
probability distribution.
Why do I say pseudo probability?
Because it is not an actual probability because neural networks are not probabilistic models
the neural networks so even if these numbers are distributed like a probability function
and you can interpret them as probabilities they are not strictly speaking probabilities.
Anyways so what we want to now learn here is two things one is we have to learn how
these vectors pervert look like and these vectors are usually called dense vector embeddings
or embeddings for short and we want to learn how to realize this kind of function with
a neural network.
To give you an intuition why this would be useful so the idea is basically a good embedding
would reflect word similarity this is a notion we didn't have in engram language models but
there are similar words there are words that behave similarly with respect to our objective
and our objective is prediction of the next word language modeling.
So maybe words like dog and cat are similar maybe words like an A are similar it is the
same grammatical function so they can be replaced in many places and maybe the same goes for
room and bedroom and is and was and running and walking so what is the intuition here
the intuition here is if you have a training example such as the cat is walking in the
bedroom it could transfer a little bit of his probability mass to similar unobserved
but yet possible sentences such as the cat is walking in the bedroom or the dog was running
in a room and so on right so this is the idea you would transfer probability mass via similarity
on those vector embeddings okay so to go a little bit into details here we decompose
this function that should return our pseudo probability in two parts one is basically a
mapping and here it's called C and it's a little bit unfortunate because C was previously used as
count and now it's the embedding matrix sorry about that but this is the original notation
of the paper I decided to leave it as is because if you want to read the original paper then you're
not getting confused okay so we have a mapping from any element in our vocabulary to these
vectors right and basically what that gives us is a matrix and the matrix has m dimensions and
this is the length of the embedding on the one hand and the number of vocabulary items which is
very long on the other hand and these numbers are free parameters so these are parameters that are
learned during the process so it is a lot of parameters right like you have vocabulary maybe
a million words and then a hundred dimensions so you have a hundred million free parameters
sounds like a lot but it's actually usually less than what you have to keep track of if you do
say five gram counts four gram counts try gram counts and all the counting required for a large
scale angram model okay so this is the part where basically words are getting yeah kind of
translated into vector representations and then we need the function it actually does something
with these vector representations in order to realize our function here so basically we model
the function on the basis of a function that takes the preceding vectors of these words and
then you can yeah put any of the next words in here and this would return you a probability
distribution over all vocabulary words that can be predicted here which should be the entire
vocabulary of course because you can always you know say anything after anything it's just
the question whether it's likely or not and technically it looks like this so this actually
symbolizes vectors and vectors with dimensions so these red dots are values and these values
could also be outputs of neurons right so if you have a layer of neurons few neurons next to each
other and they have output what comes out is a vector and yeah now let's go the step from
traditional angram modeling to neural angram modeling so first we have an angram model so
we look at the few words in the past and these are here so this is the word before our current word
this is the word two words before this is the word three n words before depends on what your n is
and here are the words and you look them up and basically produce this dense vector embedding per
word and what you then do is basically you concatenate them you put them all together so
this is like a single vector and then we put some neural network here so some amount of hidden
layers that actually learn the function and the function would basically transfer whatever we have
here and the representation to predictive values for the entire vocabulary so what we see up here
is a very long vector so this vector has as many entries as there are words in the vocabulary
and then we might you know pick the largest one if you want to generate one or have some distribution
on what is the likely word what is a less likely word given the current past the current context
yeah let's go a little bit in more details here so what comes out so let's just imagine neural
network so you know it's basically inputs and it has weights and it has some function and then it
passes on stuff so basically here a bunch of numbers come in then you copy them up then there's
a bunch of layers so these numbers kind of get transformed and then in the end there is a direct
link between the last hidden layer and this output layer and it's fully connected right so
everything here would actually have a lot of connections with a lot of weights to the previous
one and if you just sum up the product of the weights and the outputs you get some number here
the number you get here is some number but nothing actually guarantees that what you have
up here is a probability distribution for a probability distribution you would need them
all sum up to one but we don't have that right it's not never guaranteed and this is and since
we want to have something that resembles probabilities these sort of probabilities
what we apply here is the so-called softmax it's a softmax layer who has heard about the softmax
layer before okay most people it's always applied if you want to basically have a distribution over
the output layers and what you do is basically you yeah take them all to the exponent and you
divide by the sum of all the exponents and since you normalize by the entire sum this actually
gives you values between zero and ones so this is actually how you enforce that you have something
that looks like a probability up here and this vector which i call y here is actually an
unnormalized um yeah log probabilities which we then take to the exponent to make them probabilities
and normalize them and this is composed of a bias and a weight matrix that actually directly takes
some connections between these guys and up here plus some neural network which basically does
something with their entirety and transforms them and this picture is under specified in the sense
that we have most computation here which is basically uh some arbitrary neural network so
usually they use one hidden layer of like 50 urans or something like that but it could be
arbitrarily complex in here okay so this is uh a setup on how to basically do a neural version
of the engram model um if there are any questions before i lose you please ask them rather earlier
than later looks good okay um let's talk a little bit about parameterization about differences
and how to learn these things so what we have is a lot of parameters in this so we talked already
about the parameters that are created by just having this vocabulary by embedding matrix it's
full of three parameters but we of course also have parameters for the network in between and
these let's call them omega and these c parameters for the matrix and the omega parameters form the
overall parameter set of let's call it teta and now we want to train this and our training
objective is language modeling so basically what we want to do is we want to maximize
the training corpus likelihood while in fact we want to maximize the held out test sets likelihood
but we can't optimize in this directly because it's a held out test set so we do this on the
training corpus so the likelihood is actually we basically take whatever comes out of this
function parameterized by teta and we sum this in the log space so you remember summing in the
log space is like a product in the non in the normal space but summing over log props is
numerically much more stable this is why we do it and then divided by the corpus length and this is
basically kind of the average prediction probability for each word in our training corpus
plus some regularization term this is something I can't go into basically regularization is a
method to prevent overfitting to prevent total memorization of the training set which usually
results in very bad performance on held out test sets and in this paper they use something which
is called weight decay penalty so they want to make sure that there are no super high weights
where actually everything goes through so this is how a network would memorize a particular
examples but they want to have weights kind of nicely distributed and this is like a penalty
term that regulates that and again I have to point to other lectures where this is discussed
in more detail and how do we actually adopt the parameters what we do is we set the new parameters
by taking the old parameters and changing them a little bit and we change them into the right
direction and to give you an intuition so basically the parameter space
would induce something like you know a hilly landscape so where actually uphill is
better likelihood let's say and we can evaluate our parameter space maybe we are here
and we actually know what we expected right because we know if we have training data
and we have you know run this on the past and have looked at what comes out of the softmax
maybe the actual word we observed is this one and ideally this would be one and the other ones
would be zero and what we do is basically we take the derivative of the probability
by the parameter space which gives us a direction of how would we actually walk where would we go
in order to tune this more towards what we actually wanted to see in the output which is
this one at the real world real word observed word and zeros at the others but of course we
don't want to just set it directly to this one because this would yeah be very susceptible to
the last example we learn and this is why we put a little learning rate epsilon here which is a
small number like 0.00 something and this actually tunes the parameter set a little bit into the
right direction and we do this several times uh for many many iterations and hope that we actually
yeah gradually move our parameter configuration to a place where it actually optimizes the training
objective which is likelihood of the corpus okay um so this is a table in the original paper
and i decided to actually just throw it at you because it's very instructive in the sense that
you think this is a big table no so if people do this kind of experiments
and care to print a table like this in a paper chances are that they have in their back
somewhere a table of like a thousand rows or maybe five thousand rows a lot of experiments
right so this is just what they report so if you read this kind of modern paper especially in the
neural learning papers what you see is the tip of the iceberg and it all sounds so easy but you
don't see what actually has happened in the background here uh okay let's look at this
table a little bit so what we see here is also a typical um thing in research you basically want
to show that it's better so you need an evaluation measure the evaluation measure here is perplexity
and they report it on the training validation and test set and you also want to show it's
better so you compare this to baselines and baselines are usually the best previously known
kind of models here are class-based back off models i haven't talked about these but it's
basically you cluster words together in classes so that you get smaller vocabulary and therefore
more training examples per class this is kneesernei back off which is uh something i decided to take
out of the lecture because it's uh mathematically even more complex than what we saw before
but it's basically at the time it was the best sparse engram based traditional um model
here is the deleted interpolation uh and here we have different kinds of n right as an engram
and then what they do up there is basically uh different ways of the neural model so they
vary n they vary uh h which is the number of hidden units so this is where the most computation
here is going about the very m which is the dimensionality of the vectors they just do 30
and 60 here um they vary whether there are direct connections or not so this would be
in this picture whether there is a direct weight matrix from these inputs to the
uh softmax layer or whether you don't have those
and then they vary the mix and mix is basically uh in this case so they can do a normal trigram
maximum likelihood estimation model which suffers from these unseen events zeros and they okay half
of the prediction comes from this one and i just take the other half from my neural
language model why would you want to do that well nice thing about this engram model is actually
memorizes symbolically so it actually does distinguish different words whereas our neural
model maybe does not distinguish some words because maybe some of the words are mapped on
the same location or almost in the same location in the in the vector space and therefore some
subtle differences invert usage gets lost so maybe we you know just take a kind of best of both worlds
approach here and then they report training perplexity which is not interesting validation
perplexity so this is actually how you optimize your parameter space and then in the end you report
and test and what comes back here is basically what is performing best is if you in fact do this
mixing so this is actually always a little better than the non-mix version and in this case it's a
very low number of dimensions only 30 but they need a hundred hidden units and uh yeah um this
is basically the first uh neural language model uh there was this is how it was invented yeah so
it's not only neural it has this mixing kind of thing and this is actually not uncommon for
language modeling as we shall see a few slides down um maybe some general note on neural uh
approaches so it has advantages and has drawbacks so advantages uh is actually
these numbers are better right so perplexity went down from like the 300s so here was the
best classical one 312 to the 250s the best you know new one was 250 so this is like a major drop
in perplexity this is like an actually very good right it reduces like 20 percent of perplexity
this is massive um okay but it comes at a cost because with the traditional angular models you
can just count and then put them in the smoothing back of whatever formulas we had here we have a
lot of computation and the computation already starts then evaluating because whenever we
basically match the output of our model against the actual observation in order to
compute the derivative to percolate back how we should change the weights
we have to operate on probabilities so we have to basically carry out the entire softmax layer
which means that we have to evaluate it for every word in order to be able to properly normalize the
probability uh for the word we're interested in a lot of computation um and this was not
possible just 20 years ago because it requires parallel processing to scale this to corporate
sizes where it actually becomes interesting to model anything useful um yeah and then of course
there's hyper parameters so there are parameters parameters are mentioned parameters so
weights in matrices in networks and hyper parameters are the parameters that control
the overall uh process and the overall architecture so such as number of hidden
units would be a hyper parameter the learning rate you can vary the number of epochs which is how
often do you go through which of course the smaller your learning rate the more often you
should go through kind of influence each other what kind of regularization do you want to use
what is the dimension of these embeddings and so on and uh now you could just try a lot of these
but again computationally not feasible you cannot do a grid search in all the possible combinations
of hyper parameters so it kind of requires experience it's kind of a black art of hyper
parameter optimization of course you start with some brute force but then you have to
do some your own heuristic in order to find something reasonable so again if you read these
papers on neural learning you just get the tip of the iceberg they basically kind of tell you okay
so in these ranges we found good stuff but how to arrive at these ranges very hard so if you try to
do your own hands-on and the new problem um this is what you have to face
okay okay and uh maybe one more thing on neural language modeling so what this uh model has
solved is our smoothing problem and it's actually much nicer uh in the sense that we don't have to
fiddle around with this pack off or some uh mixing parameters or some thresholds or some
reshuffling of probabilities and probability mass it basically is all kind of in these embeddings
somehow um this is very nice but what it doesn't solve yet is the thing about the finiteness of
the history right so this model just like the angular models we looked at of course you could
put an arbitrary number here but it's a fixed arbitrary number and of course the more you put
there the more parameters you have to learn again because this creates a lot of weights here
creates a bigger network here uh so we have the same issue with engram as with engram models so
if you do this for too long then the training data is too sparse so this is why it usually stops at
three four or five um but we also had this concept of recurrent neural network and we can use the
recurrent neural network to actually do away with this limitation and to in principle look
at an infinite history and uh to give you an intuition um this is a model so who has heard
of virtuvac this is kind of a predecessor of virtuvac so virtuvac was invented in 2012 by
mikilov and others and this was invented two years ago by mikilov and others um so it's not
virtuvac but very similar uh and conceptually this is on the path to virtuvac and we're going
to talk about virtuvac later in the lecture but not today um so this is the recurrent version
of a language model with infinite history how does it work sorry you have an input layer
and this is just like in the other ones the embedding matrix okay so a word gets
looked up in the embedding matrix numbers are placed here then this is uh by some neural network
uh transformed transformed into the current context and this layer here would actually represent
whatever i know about the past what do i know about the past i know the last word
and i know the past of the last time step so this is another way of uh painting this dashed arrow
which is like a time delayed so you feed in the context at the time step minus one to the current
context and you add the input here now if you have a context at time step one some of it will
be available at time step two and some of it will be still available at time step three
because it was in time step two and you feed it from time two and so on so in in principle this
has like infinite history can memorize the entire context of whatever you have seen of course it
doesn't memorize it verbatim but it memorizes it somehow convoluted in this uh representation but
the information can still be there yeah and from this context which basically defines what you've
seen before you have again some neural network with again an output softmax layer over the next
word and again you train this um just like we trained the other one by for example stochastic
gradient ascent yeah and uh again this was evaluated in a mixture situation so again this
model alone actually did not improve perplexity over the baseline but uh in combination with a
this is knizane five gram language model it actually did and they tried different things
in their this paper so they tried different vocabulary sizes from 200 000 words to 6.4
million words so the embedding matrix is actually 6.4 million words long
and they measured perplexity and perplexity goes down if you add the recurrent neural network
so it's like consistent nice improvements and what they also measured here is word error rate
because they put this new language model into a speech recognition system so speech recognition
systems take the sound and they transform it somehow into a phonetic representation
and there are words that sound similar right like uh honey money sunny or whatever right
and a language model is used to rescore them based on the context so you could use a language model
to basically say okay i have this you know bunch of words that i have some likelihoods from the
sound representations but i also have likelihoods from how do they actually fit into the context
and this is where the language model is used and there you can measure word error rates on
uh transcribed test sets and also here the word error rates go down um maybe one comment on this
so we have seen here perplexity values which are in the range of 150 300 and before we
seen perplexity values in like 300 200 are these comparable no they're not comparable because
uh in this case they evaluate in different test corpora so i the ones before they ever
laid on brown corpus and this is wall street journal corpus which is different um there's
doesn't make sense basically to look at these absolute numbers sometimes you'll see uh badly
written papers or badly written master's thesis where they do related work and people just report
okay and they uh get you know 95 percent score and they get 98 score and this is why the 98 ones
are better uh not necessarily if it's not the same corpus is not comparable uh you sometimes
read stuff like the perplexity of english is 80 what is english stupid no possibly on this
particular corpus might be 80 but this doesn't tell anything about the the language in itself
right so a caveat here never compare numbers that are not comparable and they're only comparable if
you like evaluate on the same conditions and in this case on the same corpus okay um a few last
words on neural language models so basically what we did here is you have transformed what we could
call symbolic units so words um into continuous representations into these uh embeddings
and you could do this in a good way or in a bad way uh but if you do it in a good way then uh
similar words have similar representations because then they can transfer these probability
masses to each other and uh have you know generalization smoothing and all these good
properties and what does uh similar mean similar means similar with respect to the training
objective so if you train a neural dense vector embeddings on language modeling they're good for
language modeling it doesn't follow that they are good for information retrieval it might be but
it's now not guaranteed right so you want to always train them on the objective that you're
actually interested in what we see is uh they have generally better performance than sparse
angram models and it's also more compact than the model application so it's easier to fit those
guys on say embedded devices smartphones whatever but it's harder to train them so it's uh the
training actually takes place in large paralyzed tpu server tpu modern architectures very expensive
stuff uh because the training is more expensive but also because the hyper parameter search
makes it extra expensive because for every hyper parameter configuration you have to
retrain many epochs maybe um and yeah what is the state of affairs so these are now the standard in
nlp uh because the natural language processing uh has largely moved to neural methods and for
neural methods you need vectors or like numbers you can't deal with symbols really uh so uh yeah
this is why everyone is using these and current research directions actually go towards
contextualized word embeddings so what we have talked about here are um we have a word and we
look up the embedding right but some words have different meanings in different contexts we had
this uh riverbank versus money bank example in the first lecture and you would actually want
to represent these two banks differently according to the current context you observe them in and
yeah this is what modern approaches like elmo bird and gtp are doing and i'm sure next year i can
make this list even longer because it's a very active field of research right now
active field of research but dominated by the big players because you need so much hardware to
actually train these large-scale models that only google facebook apple and amazon can actually do
it okay so this is all i have to say about engram language model and neural models are
there any burning questions does not seem to be the case so we continue uh after this little
excursion on neural model methods back to more symbolic modeling um hidden mark of models
and actually this lecture is not about just telling you the latest stuff there is but the
lecture is about making you understand how to arrive at this later stuff right so
i had to walk you through all these engram models in order to make you understand
what's so good about the neural language models and i have to walk you through hidden mark of
models in order to make you understand what's so nice about conditional random fields um so yeah i
had comments in the evaluations you do too many old methods uh but i don't think so i think you
have to understand these old methods otherwise you cannot appreciate the new ones and then you
underestimate their complexity okay who has heard about hidden mark of models before
uh okay very good um so we can do this who has not heard about okay one two okay of course we
do this but maybe we couldn't do this a little quickly okay so you remember mark of chains
because i talked about mark of chains in the last lecture uh what was mark of chains we basically we
count engram and we normalize these two probabilities we have a weighted finite state automaton
and yeah we had the problem of sparse data many engrams not in training and then we talked about
this back of smoothing and the back of smoothing can be realized by a mixture model for example
so you have in a trigram linear interpolation model uh if you want to predict the next word
on the basis of the two preceding ones you could do this by predicting it on the basis of the
unigram probability the bigram probability and the trigram probability whereas you have
these little lambdas here that actually appropriately mix them together such that
it still stays a probability so this is the condition back here and we can use now hidden
mark of models to actually model this mixture directly and also to train these lambda weights
in this case and this is just one example of hidden mark of models and they're called
hidden mark of models because there are hidden states and uh for our uh formula here it could
look like this so we could actually say okay we have maybe in our example language of a's and b's
our state where the history is a b and now next we can say add an a or a b so the next
states can be either ba right because this is the b here right and then we say an a
or the next day can be bb because we have a b and in the traditional mark of chain you would
just directly connect this state with this one with uh you're uh accepting or generating a letter
a and with some probability and you will connect these directly with the b and what we do here is
we say okay now we want to model this mixture so we introduce a set of hidden states so let's call
them lambda one two three and we have epsilon transitions with some probability that according
to our lambdas and then we can be either here or here or here and depending on where we are
we take the unigram probability the bigram probabilities or the trigram probabilities
yeah and this is now suddenly a non-deterministic weighted finance state automaton
because we have these epsilon transitions so while we know that if we are in this state
and if we see a b we're going to be in this state after we don't know how we got there
and depending on how you want to model it it's you know you either take one of these or you can
take them all at the same time maybe um but we don't know the path and uh this is why it's
called hidden because the sequence of states in this model is not determined anymore by the
input symbols or by the emitting output symbols you can always generate it as well yeah and
this is the formal definition of a hidden Markov model so it's kind of like a fine state machine
so you have a set of states you have an initial state probability distribution to know where we're
going to start we have a finite alphabet of input symbols and we have a transition function so we
are in a state and either we see a input symbol or we see nothing then we have a weight function
between zero and one whereas it's normalized per state to say it's a probability and then we go to
the next state yeah and yeah it's non-deterministic and what now the and it is a probabilistic model
so as opposed to these neural models it's actually really probabilities and if you talk about
probabilities the question is always how is it normalized and it's actually normalized over
all possible combination of state sequence and symbols so if you want to sum up stuff up to one
you would actually have to generate all the possible sequences and for each sequence
all the possible paths that you can take through this network through this
weighted finite state automaton and this in its entirety would sum up to one so it's normalized
over all possible sequences and all possible paths that can be taken for these sequences
which of course means that the probability of a single sequence is usually very low
but we don't care as much about this just as in angram language modeling we use this for
comparing stuff what is more probable than the other one the absolute number doesn't really
matter so much okay so how do we actually now compute the probability of a sequence
so we have the sequence of input symbols s1 to sn is actually summing over all possible
uh state sequences that are possible that we can accept this sequence what does that mean
basically we have to see okay we are in state at time step one and if you are in state in
type step one then there's a conditional probability of accepting the first input
symbol and going to state at time step two and once we have that there's a conditional probability
of going to the state at time step three by accepting the second symbol in the input symbol
and once we have that and so on blah blah blah blah blah so this becomes lengthy okay
and now we use uh the Markov assumption the Markov assumption was just as in the Markov chain
limited horizon the next state is only dependent on the last state and the input symbol and not
on like a list of last states not on a longer history you basically the history is the state
nothing else one single state so uh this is why we can actually shorten down these conditional
probabilities here to just the last state right so for knowing the probability to go to the
second state in the time uh by accepting the first symbol and the sequence we have to know
only the first state and for uh computing the probability of going to the last symbol
in the last state in the state sequence which is one more than the number of symbols because you
start with a state and you end with a state and between you accept symbols you only looked at the
state before that so this is the Markov assumption and then we can write this as product which is
basically the product over uh all of these in different steps in time and then summing
over all possible state sequence because mind you it's not always the same state sequence
there can be different state sequence even for the same input and uh yeah we can also do a different
notation here so basically uh we go on from this state with this symbol to this state
and this is basically the same just a different notation here yeah and uh this is basically
the probability of a sequence and of course we want to uh have a good model that models
known sequences in our training data well uh specifically we have three tasks what we want
to do with hidden Markov model so uh given a model uh defined by an HMM how do we efficiently
compute the probability of the observation sequence so that focuses on efficiently because
enumeration of all possibilities is intractable then second task given we have this observation
sequence and the model um how do we actually choose the state sequence that best explains the
observations because maybe we want to use the sequence of states as labels in some subsequent
process so now we're interested in the the best path somehow and the third question is yeah how
do we actually get to the model so we have an observation and we have a space of possible
models how can we train the model how can we actually set these weights are in these
transitions between states such that we maximize again the likelihood of the training
okay um let's look at how we compute the probability of some observation so basically
we have the sequence of symbols and of course we can then generate all possible state sequences
and then we have all possible paths and then we look at these paths and multiply all these
transition probabilities and then sum over all the paths and yeah then we have an overall
probability of the sequence this is yeah works mathematically but it's not efficient because
if you have like n states and t is the length of the sequence then the complexity is t times n to
the power of t just because you have so many paths and you have yeah and for every situation
you have n to the power of t and you have t times the entire thing so it's it's too crazy okay um
and that's why we need something more efficient and again so you have heard about this but who
has heard about forward backward and vturby already okay much less people okay so let's do
this a little bit more detail um so now the quest is how can we actually do this efficiently
and this is a dynamic programming concept you remember dynamic programming from
the algorithms and data structure lecture or similar lectures where you actually intelligently
reuse partial results in order to compute the entire solution and you reuse them you don't
recompute them all over again so uh here what we're using is we're using common sub paths and
combine them into longer sub paths and for this we have to uh do a few definitions just bear with
me so there's a concept that's called the forward probability so we call alpha um at time no alpha
for state index i this is not a time index but this is like the if you enumerate these states
the state index alpha i at time step t this is the probability
if you have seen the sequence up to t minus one
so that in your state at time t is the state with the state index i
so the probability of finding yourself in a particular state z i at the time step t for a given
sequence and uh yeah if you then use this concept and you do this for the entire sequence
then you will get for the time step up case t plus one which is basically
once you're done with the consumption of the entire sequence you will get a probability of
being in each state after you have consumed uh the entire input and likewise you could do this
in a backward fashion so we can call um better i t uh something yeah better i t the probability
of finding yourself at time step t in the state i if you go backwards if you walk
all the paths backwards and therefore you're looking at the sequence from the last
symbol in the observation sequence up to the symbol that is currently um yeah at your time
step of interest okay so these are just definitions and now let's do the forward procedure so the
procedure goes as follows what is the probability of being in a state before we actually consume
a symbol well this is actually the start state initial distribution right remember we had this
pi which basically gives us a distribution to use a particular state a start state
so at time step one we haven't seen any symbols yet this is actually for all states the forward
um probability and then we have an induction step so if we want to learn about the probability alpha
and the forward procedure to be in state with index j at the time step t plus one
we have to ask ourselves how can we get there well we have to use a transition to our state with
index j and of course we have to use a symbol which is currently observed which is the symbol
at time step t and we could come from all other states or all states you could come from the
same state as well we can come from all states and what is the probability of being there well
this is just the probability of being at a given state for the time step before
and now if we sum over all possible states uh to get at this new state in the new time
this is what we this is how we basically fill our table of uh forward uh probabilities so here's
a typical dynamic programming thing where you actually use previous things for the next time
step yeah and then what is the actual results so mind you we wanted to have the probability of the
entire observation sequence giving the model um yeah the probability of the sequence is then
we look at what is the probability of being in each of these states at the last time step
and then we sum over all these states and then it's basically all the paths
and if we look at how many computations do we need so basically we so this is the
expensive thing here right so what we do here is basically we have to loop over all states
for a single index here and of course we have to do this for all indices so this is actually
quadratic in the number of states and how do how often do we have to do it well as long as the
sequence is so this is why the complexity is t length of sequence times n square so uh number
of states to the square and this is of course much better than the exponential explosion that we
seen in the naive case yeah and uh if you want to do it backwards we can do it backwards
yeah same thing just uh be careful if you look at this at home uh that the initial state
distribution is now here and not up there but small detail okay and we can actually combine these
so actually we could also write the probability of sequences by summing over all possible states
in all possible times coming from the forward and from the backward yeah um okay so this is basically
an efficient way of computing the probability of a sequence what we also want to use this for
is what is called decoding and decoding is whenever you have an input which is a code
and you want to find some structural information on it which is like decoded um so in translation
you have decoders um you're going to see an application of hmm's just in the next chapter
which is part of speech tagging which basically assigns kinds of words for words in the sequence
kinds of words just as noun verb adjective these kind of things and we're going to associate this
information with the actual states in the hmm so decoding is finding the best state sequence
so what sequence through our hmm is actually the most likely one given the input
so formally what we're going to do is the following we want to find the maximum path
which is the argument of paths so this is like a state sequence
in times that maximizes the probability that given we have this input sequence
s1 to st that this is actually the the the maximum path uh like this sequence and this is actually
if you're in the argmax space equivalent to the joint probability so this is a conditional
probability and you could rewrite this with the bayesian rule into a joint probability
divided by a factor which is the same for everyone which is the probability of the sequence itself
which we're not interested in so if you're doing argmax basically it will still maximize the same
thing so we we can look at the joint probability here and now we do a dynamic programming solution
which is called the viterbi algorithm who has heard about viterbi algorithm before
yeah this is something you usually learn together with hmm's which actually
gets you effectively and computationally efficiently to the maximum sequence and for
this we define our little delta no is it the delta yeah small delta uh delta at uh for every
state index i at time step t which actually does not record the probability of being in this state
so it's not alpha it was the probability of being in the state at the time step
but the probability of the maximum path to this state so in this alpha case and the forward
procedure we basically looked at all the possible paths to get there and for this delta we only look
at the maximum path and therefore it's not summing over all possible state sequences but
it's maximizing it such that at time step p we're looking at this sequence of states
and this input sequence and we're interested in being in the state with the index i at the time
step t and we want to have the maximum one and this is the viterbi algorithm very similar to
the forward procedure just that you don't use a sum but a max uh so let's walk through again um
delta at time step i uh time step one for state index i so we haven't seen any
sequence yet so this is basically the initial state um distribution so uh this basically
follows from our pi i for each state and then now if you're interested in the probability of
the maximum path leading to state j at time step t plus one we look at the maximum paths to arrive
at every state i we look at the probability of going from state i to our state j we're interested
in under consumption of the symbol at time step t and we decide on taking the maximum one
basically just memorize the maximum path and so that we don't forget what the maximum path was
we um we store a so-called backtrace matrix so in this uh uh phi j here we basically
memorize where we came from so we memorize uh what was actually uh the state
z i we came from in order to create the new delta j
we're going to have an example in a minute where we actually see how this is executed and why it's
important and then let's look at the end of it so basically we uh want to know okay what is
the maximum path going over a sequence of states so what this computes is basically for every
possible state in our list of states what is the maximum path per state and uh at the last time
step we basically take the largest value we find so the largest delta i for all i's and this is
basically our last state and then we can read the sequence backwards because we memorized it
and actually multiply out the probabilities and emit the probability of the state okay
in case you wonder about ties about you know stuff where if you take the maximum
maybe there are several ones that have are equally good um yeah there's different ways
of approaching this so sometimes people store and best lists some people just resolve this
randomly for most nlp application actually it doesn't really matter so much like in the end of
the day okay um let's do an example so this is a hidden markov model uh hidden markov model
what what is hidden about it well we have two states one is called n the other one's called v
and it's definitely the case that uh if i tell you a sequence of a's and b's this is our language
uh you cannot tell me what was the path i used to generate it um and we have the initial state
distribution which is basically this is the start state so this is makes it simpler okay
and uh yeah so what basically uh uh how do we determine now what is the likeliest
sequence of states for a given uh input let's go okay so observation sequence is b b b a
and uh for this we basically have to keep track for each state in every time step
what is the maximum path to get at this state so for the first time step we haven't read any input
the probability of being in n is one the probability of being in v is zero
now we have read b we can go to n or we can go to v
and we can come from n
or we can come from v and what we record here is always the maximum that we end with n
and here's the maximum that we end with v so how could we go from with a b to n
so we can go with a b either like this to n or we can go with a b to n like this
this has probability 0.1 the probability of being in v before is zero so this is zero okay
the probability of being in n before was one the probability of then using this one is 0.2 so this
is why the max path after this one to an n is 0.2 likewise let's look at how can we arrive
in a max path to v well we could either have walked from v to v so this would be zero times
0.5 or we could have walked from n via a b to v which is one times 0.1 which is larger than zero
this is why we put the 0.1 here and this is the maximum sequence okay then we go on so if you see
the second b we could either come from here or from here and what we basically do is okay
we do this times the transition probability of n to n we do this times the transition probability
of v to n and we take the larger one and in this case the larger one is 0.2 times 0.2
which is 0.04 which is why we decide on copying the n here and the last n is always given because
this is the n sequence it ends in n and likewise we do the same thing down here and we can do this
for all the steps and yeah maybe what is interesting like up to here it looks like this
n sequence is always like more probable and then it kind of turns around so in the end these two
sequences are actually equally likely and i'm not going to walk you through all the steps now you
could do this at home but the the the the capture idea the important idea is basically to realize
that we always look at how like where do we end uh what state are we arriving and there's a max
path to this state uh during the process and why is this more efficient because we have this
search space if you like would enumerate all the paths then we would actually have this giant uh
tree of possibilities and uh what vturbi actually does is to cut hopeless paths
from further exploration so uh whereas in this uh inefficient case of uh computing
the maximum path by enumerating everything what we would have is like a full you know in this case
binary tree is just binary because we have two states we have more states that would
forget even more and it would be the full binary tree and this is why it's exponential
and what uh vturbi does is basically keep this always down to two previous
or like number of state previous possibilities explore states by states and then prune again
and and and pick the maximum one so this is why it's computationally more efficient just to give
you a different perspective on that okay are there more questions for forward backward and
vturbi no okay yes so is the determine from the result of the term algorithm is always
smaller than the result of the backward or forward algorithm right yes smaller or equal but usually
smaller yes yeah yeah okay let's do hmm training so what is training here so we observe we have a
fixed structure so the number of states is fixed right this is a hyper parameter uh and now we want
to optimize the parameters so the parameters are the transition probabilities on these arcs
according to the observations and what we're going to look at is now bomb well so forward
backward algorithm training and uh the idea is basically an idea that we also see with other
training algorithms this is why i do this a little bit in more detail because it's still
understandable here um is that you initialize them somehow randomly and try to improve them
and uh i didn't say this explicitly but in the neural language modeling
yeah you also have to start with some parameters so you initialize them randomly right um
and uh yeah what is then improve them you basically have to have a procedure that actually
is guaranteed to uh converge to something better and this could be either because it has some
mathematical properties to converge to something better or because you do a trial error approach
you could also say okay let's randomly wiggle a little bit let's you know change them a little bit
see if it's better if it's not we go back right this would also converge to something better in
the end although it's not very directed um and of course the question is why is this necessary
because i mean in the mark of chain it wasn't necessary right we could just
count the number of transitions number of n grams number of conditional probabilities
why do we have to do this random stuff here and for this um we have to take a look uh at these
formulas again so let's recap um this is a mark of chain why is the mark of chain because it's
deterministic so you never have uh two possibilities uh for the same input symbol
um and if you now fix this architecture let's fix it like this and we have a training sequence
what we basically have to do is we just have to count how often do we walk which path
and yeah it's deterministic so we can see what the path was and then we can normalize appropriately
right so how do we train this basically we run the sequence in this case a b b a a b a
through this thing and then we observe okay we went from n to v using a five times we went from n
uh to n using b three times these are all the outgoing arcs of n so these are eight right so
that's then call this five over eight and let's call this three over eight and likewise outgoing
arcs of v uh we have a total of four and two times we used um this one and two times we use
this one this is why they're both the same probability of two over four or 0.5 um yeah
but with hmm we can't do this because it's not deterministic so how do we do this if we have
you know multiple possible paths are we using like do we assume that all possible transitions
are used at the same time and how do we actually start the process because yeah uh let's look at
this a little bit more closer um let's simplify the problem let's not make this arbitrary all
possible whatever but let's focus at like particular uh transition in a particular path
so maybe we have only two possible paths through an hmm and let's say the first path has a
probability of one third using a particular transition t one and the second path has the
probability of two thirds using a particular transition t two and now if we observe
that we have these inputs and we have these two possible paths we could actually increase
the count of this t one by one third because the probability that we took it was one third
and we can increase the fractional count uh of the t two by two thirds
makes sense yeah why not right we could do fractional counts who cares doesn't have to be
integers and the likelier it is that we take it the more it gets increased if we actually observe
something that would actually use these paths okay uh in the general case it looks like this so we
want to count the transition from state i to sound state j under consumption of a particular input
symbol and there you would actually have to sum overall the state sequences that you can
take in order to consume a given input sequence and then you look at how often you actually take
this particular transition so this actually gives you the probability of uh the particular paths
and this one gives you how often this transition is occurring in this path and uh
uh now that you want to count this you have to normalize by the
probability of the observation itself in order to get the joint probability
so if you want to take this out of the condition and make a joint probability you have to like
normalize this out probability computation rules um yeah so let's look at this again the count
is actually determined by how of how probable is the path and how often is the transition in the
path and for the probability of the path given the uh input sequence we have to normalize by
the probability of this input sequence and uh yeah so actually what is um
um the probability of a transition well the probability of a transition is actually what
we do is we count how often we saw it and we count how often we saw total outgoing transitions
from state i which gives us this probability distribution of outgoing edges on the c1
and now it becomes circular because for computing the count we have to compute the probability of
the path normalized by the observation sequence time so often we saw it the
probability of the observation sequence is actually we know that right it's the forward
procedure so we can sum over all possible state sequences and then do the multiplication of all
the probabilities on the single time steps and for this we need the transition probabilities
and the transition probabilities are actually defined as how often have we seen this one as
opposed to all other outgoing ones and there becomes circular right and this is why you have
to kick it off somehow and this is why you need this kind of uh random initialization and the
procedure that makes sure that you actually go from your random initialization to something
more useful in some iterations and in this case uh the algorithm used for this is called expectation
maximization who has heard of expectation maximization before okay very few people okay so
idea is basically it's kind of a bootstrapping so you basically it gets you out of the swamp on your
bootstrap right so what you do is basically you kind of want to
yeah converge to something that's better by iterative time steps and on top level expectation
maximization looks like this so basically what you want to optimize is cross entropy so basically
a measure for how well your model actually models the sequence so if you call this cross
entropy or likelihood conceptually same thing and that's called the old one is infinite and you
want to lower this right so like lower is better for cross entropy and then we guess the parameters
we basically roll the dice and you know random initialization and then we have some re-estimate
parameters function and as long as we don't get more improvements so as long as we still get
improvements as long as the the new cross entropy is not kind of like the same as the old one
we basically memorize the old one as what was the new one and get a new one by
re-estimating the parameters and computing the cross entropy and now it's not enough to say okay
we have this problem and then we use expectation maximization what you actually have to do and
this is what the big thing with Baumwelsch was is actually to prove that the method that
you're using here is actually guaranteed to lower the cross entropy
so if it's guaranteed that this re-estimate parameter step lowers the cross entropy then
the algorithm converges to a local minimum or minimum maximum optimize
towards zero towards whatever right it's interchangeably yeah and yeah it has been
proven and this is why it works but i spare you the proof i just show you actually how it's done
on an example so let's say this is actually the true HMM we're cheating now we know what
actually the result should be because we used this one to generate a sequence and we generated a
sequence a b a b b it's a very short sequence usually you train this along the sequences but
whatever for the small example and we can also initialize the same architecture randomly so what
we have to set is all the free parameters and actually there are three free parameters
this one this one and this one this has to be one because it's the only outgoing arc
so you don't have to like put random numbers everywhere you have to put random numbers
wherever there are still free parameters and also these three guys actually are depending
on each other right because the sum is one so you basically would have two random ones
and the third one would basically follow from that okay and um let's say we have a training
sequence a b a b b and for the sake of simplicity we're now enumerating all the possible paths
which is not how it's done because of course enumerating all the possible path for large
HMM's and large number of input symbols whatsoever will become intractable again but in this case we
can because it's just four we can either go n b n v and n or we can not go via v
with this a b combination so every a b allows you basically to go to v and back
and but you could also stay in n right so this is why for a input sequence with two ab's you can
either go to v once or twice or not and if you go once you can decide whether you go with the
first one or with the second one and this is what's reflected here and uh yeah what we do is
we basically uh carry out a procedure where we uh um yeah look at uh the current probabilities
of these paths and do fractional counting of how often do we visit stuff according to the
current probability of these paths so basically we're using this idea of fractional counting
and we start randomly and then we renormalize and this actually gives us
an improved model and i spare you the proof that it actually really always improves but it does
okay so what do we have to do here we have to look at all the paths
and we have to compute the probability of the path which is simple we just basically
go like okay this is our path and v and v and n so and
then this times this times this times this times this result is this number here okay
and we do this for all paths and then we look at the probability for all the transitions
and the probability for all the transitions is actually the number the probability of the path
times the number of times we took this transition so um the probability of going to v under observation
of a and being in state n is uh well for this path n v and v and n we do this two times right
we always go from n to v and we see the a we do this here we do this here um so training sequence
was a b a b b remember so this is basically two times this one okay in this path we see only ones
and this times we see it zero times this is why zero times this one is zero okay and we do this
for all the possible transitions and now we sum over all paths so the current probability of the
sequence under the given model is the sum over the probabilities of all the paths which is the
sum of this one and uh yeah this is actually uh we could also sum this up for these so this is
the entire probability mass that is actually assigned to each one of them and this is not
a probability because we have to normalize this to make this a probability so we have to normalize
so that all the outgoing edges and now we're totally talking about these guys here actually
sum up to one so what we do is basically we sum up everything that ends no that goes from n so this
one this one and this one and we divide these by the respective sum and we also do this for
the v state but this is always trivial because this is only one and we divide it by itself so
this results in one and this divided by the sum of everything that goes out of n is this
this divided by this is this this divided by this is this so what we get is basically again
a probability distribution outgoing edges from n sum up to one outgoing edges from v sum up to one
and these are actually this is the new uh probability we're gonna set for these transitions
so we have 0.06, 36 and 58 and we started with
0.04 and 48 and 48 so it went to some other direction and we can do this again and if you
do this again then we get a bunch of numbers down here same uh computation and so this transition
here actually becomes larger it goes to 0.1 and this one gets lower and lower and this one
also gets a little lower so basically uh yeah it adjusts and what happens is that the probability
of the sequence actually goes up so the probability of the sequence for our initial random estimate
here the probability of the sequence was uh where do we have it yeah it's the sum of all paths so
this is the probability of the sequence the probability of the sequence for the next iteration
is already higher if you want to do this at home the probability for the next one should be this one
and uh yeah it gradually converges to some local maximum
but it's a problem it's only local uh it can't decide for critical points uh so there are
situations where it doesn't know where to go and get stuck uh small training sequences are a problem
and uh it's a local greedy search not a global optimizer but uh i think in the interest of time
i have to stop here and continue next week are there any more questions immediately for this one
okay then thank you very much see you next week