Thank you.
Good morning, everybody, and welcome to the lecture on estimation and inference.
Are there any questions? Please raise your hand.
It seems that there are no questions, so I will briefly repeat where we left off on Tuesday.
We are currently in a part of the lecture which is a little technical, as you have noticed.
And I'm not going down to the greatest detail of technicality, but at least on an,
well, somewhat more than intuitive level, I would like to explain to you how econometric
arguments are made and what kind of properties can be established with what kind of assumptions.
And in order to understand that well, when we talk about asymptotic properties, as we do here,
so about properties which estimators will attain when the numbers of observations
increases greatly, so that it is really a huge number of observations which we can employ,
basically, in our imagination approaching infinity, and then we can have these type of
properties under certain assumptions. And I go back to the setup here, which we studied already
last week, where we actually started out from the normal formula of the least squares estimator,
of which you know that we can write least squares estimator beta hat as the true parameter beta
plus x prime x inverse c x prime u. So in order to have an unbiased estimator,
we would need an assumption which implies that this x prime x inverse c x prime u is equal to
zero. And we know that our assumption A2 is precisely the assumption which we need, namely
the assumption that the matrix of regressors is strictly exogenous to the vector of errors.
However, this is a very strong assumption, and for many data sets that may just not be true,
I will explain this a little more detail a little later, why this is such a problematic assumption,
assumption A2. And so we look for a weaker set of assumptions. I ask here on the slide,
what happens to the least squares estimator if A1 to A5 are not satisfied. Actually, I will just
talk about assumption A2 not being satisfied, or assumption A5 not being satisfied, or the two of
them. But I will not really discuss what happens if assumption A1, which is a completely harmless
assumption, or assumptions A3 and 4 are not satisfied. Our focus is on A2 and on A5. And
first we talk about A2, about the assumption of strict exogeneity of the x matrix. Now,
as I pointed out last time, and as I had pointed out already in the previous lecture, x prime x
can also be written as 1 over n times the sum of small xi times small xi prime, where the xi
is the observations we make for the ith regressor in terms of explanatory variables. So all the
variables which explain or may potentially explain the behavior of the ith regressor of our
n, sorry, which explains the observation on our ith observation, out of n observations which we have.
Travel speaking clearly today, apparently. Once again, i is the number of the observation,
and xi collects the regressors, which potentially explain the ith observation, out of n observations,
which we have. So xi is a column vector, and xi prime is a row vector. When we multiply xi
by xi prime, we get a matrix. And this matrix would be then obviously k by k matrix, because xi
is a k by one column vector, and xi prime is a one by k row vector. So that's a k by k matrix,
which we get here, exactly like x prime x is a k by k matrix. Now, x prime x is equal to this sum
here. Sum over xi times xi prime, the factor one over n is actually not of course included in this
x prime x matrix, but the factor one over n came in this in these parentheses here, just by dividing
by one over n here, since this is the inverse of one over n is actually equal to n times one over n
that cancels. And the sum over xi xi prime is precisely equal to x prime x. The only reason
why I wrote this in this more complicated way here, you know, I'm usually supportive of writing
everything in matrices, because it's much easier than all those summation signs. But the only reason
for me to write it as a summation of a vector product here is that I want to show to you that
one over n times the sum of xi xi prime is actually just an average which we take, right?
It's the average over all those matrices xi xi prime. We add all of those matrices up,
we have n terms to add up here, and then we divide by n. So this is an average which we compute here.
And the same thing happens here with xi times ui. Again, we have for n observations,
n terms of the form xi times ui. I add them all up and then I divide by n, so this is just an average.
The argument we want to make, and we have already made in the last minutes of last lecture,
is that these averages converge to something. Actually, what we would like to show is that
this matrix average here converges just to some finite matrix. Any matrix, the only requirement
being that it is invertible, and that this here converges to zero. So then we would know that some
matrix times zero is zero, and this would imply that the limit, the plim of beta hat, would be
equal to the true parameter beta, so we would have established the consistency of the least
squares estimator. That was where we left off last lecture. Now, I define what a consistent estimator
is. Recall, consistency means that the plim of some series of random variables converges to
a certain limit, which we denote as x here, then x is the plim of xn.
I introduced you to the law of large numbers, which essentially says that under certain conditions,
the sample average has as its plim the expectation of the random variable, which
we have underlying our samples of which we take the average here.
Now, this law of large numbers doesn't help us directly in our estimation setting, because the
law of large numbers actually implies that we add up observations which are independently
and identically distributed. That need not be true in the regression context, but there are
weaker forms of the law of large numbers, which have the same implication, which then allow also
for some type of weak dependence. This is one of the technical details which I spare out of this
lecture. Here, I just inform you that such conditions exist. So under proper conditions
of weak dependence, the sample average of a series of random variables, which are not
identically and independently distributed, at least not necessarily so, also converges
to the expectation of the random variable, which I denote here by new z.
And then essentially, we left off at three different possibilities of assumptions,
which would still deliver the preconditions for this weak law of large numbers.
One assumption would be assumption 6.1, where we would require that the conditional expectation
of the error term ui given just the observation on regressor xi is equal to the unconditional
expectation of ui. And this is, of course, just zero by assumption one. So we would only require
that the observation xi is strictly exogenous to ui. But we would not require any more that the
whole matrix of x's is strictly exogenous to the whole vector of shocks in our model. So this is a
much, much weaker assumption than assumption A2. And in this case, we say that the regressors xi are
predetermined. So the xi, you can think of as the xi realizing before the ui realizes
the xi's are not affected by the value of the xi. And the ui then realizes without
any reference to what value xi has taken on. So there is no feedback mechanism from xi to ui either.
Another alternative assumption is assumption 6.2. And this is actually a weaker assumption
than assumption 1. So it is implied by 6.1. This assumption would say that for each observation
i, we have the property that the expectation of the product xi times ui. So actually the covariance
of xi and ui under assumption A1 is equal to zero. And in this case, we say that the
regressors and the error terms are contemporaneously uncorrelated. So note that 6.2 is implied by 6.1.
And moreover, it is the case that both 6.1 and 6.2 are implied by assumption A1.
So by assumption A1 along with the new assumption A6.3, which would say that for each observation i,
xi and ui are just independent. So that's the strongest of all the assumptions, right? So
just put it in the right order. A6.3 implies A6.1 and A6.1 implies A6.2. So the weakest of those
assumptions is A6.2. All three assumptions state in slightly different terms how disconnected
the regressors xi are from the errors ui. Now, what do we need this for or better, what do we
need to have the desired property that the CLIM of the least squares estimator is equal to the true
value of the parameter. So it's equal to beta. As I pointed out just a couple of minutes ago,
this is the formula for our estimator. Beta hat is equal to the true parameter plus the product
of these two terms. So obviously it is sufficient for consistency to have that the CLIM of this
first matrix here, 1 over n times the sum of xi xi prime, that this is some finite and invertible
matrix. And I call this sigma xx here. I don't make any assumptions about this matrix sigma xx
other than that it be invertible. Of course, we do need the invertibility because we have to take
the inverse here. But apart from that, sigma xx can be just any matrix. The only requirement being
it must be a finite matrix. Why must it be a finite matrix? It must be a finite matrix because
we want to multiply it by a matrix whose limit is 0. And obviously, a finite number times 0 is
always 0. However, if one element of this matrix here were to converge to infinity, then infinity
times 0 is not defined. So we cannot be sure that the product is 0 or that the product would converge
to 0. This converges to infinity and this converges to 0. There are many things which can happen,
right? So we may be in trouble in this case actually. Therefore, we do need the property
that this matrix here is finite. And we'll come back to this finiteness property a little later
and you'll see how important it is. All right. So assumption A7 would say,
well, this is a finite matrix. It is invertible. So the inverse of this matrix is also a finite
matrix. And then assumption A8 would say that this matrix here, basically x prime u, converges
as the number of observation goes to infinity to 0. So the plim of this matrix is 0.
So there are two assumptions about the plimps of sequences of random variables. One assumption
says that the plim of x prime x is equal to sigma xx and that this matrix is invertible.
And the other assumption says that the plim of x prime u is equal to 0.
Now this assumption A8 is actually the decisive assumption which we need to get the property of
consistency for the least squares estimator, right? Because we do need that one of these
matrices here is 0 asymptotically, right? Converges to 0 asymptotically has a plim of 0.
Then it's not so important anymore attempts with the other matrix except for the case
it must not become infinite. So the decisive assumption is assumption A8 here,
which guarantees us that the second matrix here becomes 0 asymptotically and then 0 times
something which is finite is equal to 0. So all these ugly terms here just fade away
asymptotically and we have the asymptotic result that the least squares estimator is equal to beta
asymptotically or converges to beta as the number of observations increases. So we would know when
we have sufficiently many observations that then we are probably quite close already to the true
value of the parameter vector. Now this assumption A8 here is actually implied by assumption A1,
which says just that the expected value of the error term is 0, so harmless assumption,
and assumption A6.2, so the weakest of the three assumptions which I have explained before.
It suffices that the correlation or the covariance between xi and ui is equal to 0,
so it suffices for consistency to have uncorrelatedness between the error term and the regressor.
Conversely, we are always in trouble when we see that the regressor correlates with
with an error term and you will get this as an exercise actually to show that this can happen
very easily in simple macroeconomic models. So when we have assumption A1 and assumption A6.2,
then we can use a weak law of large numbers to show that then assumption A8 votes. It is
actually implied by A1 and by A6.2. Now perhaps some comments on what the difference between
these assumptions A2 on the one hand and A6.1, 6.2, 6.3 on the other hand is.
Assumption A2 postulates that the complete x matrix is exogenous and therefore it is uncorrelated
with a complete u vector. So when you think of let's say macroeconomic data, you will
basically never have assumption A2 satisfied because macroeconomic data are very often
some kind of time series data or some type of cross-section data. And in both cases,
assumption A2 is not likely to be satisfied. Think first of time series data. When you have
time series data, then typically what happens in an economy in some period t minus one
affects what happens in the economy in period t. Certain decisions made in period t minus one
will almost always influence what kind of magnitude the variables in period t take on.
So if you have a shock which hits the economy in period t minus one, then you will almost
certainly see that this affects the regressors in period t. And when we have this dependency
between regressors in period t and shocks in period t minus one, then we have already a
violation of assumption A2 because then it is not true anymore that the complete x matrix is
exogenous with respect to the u vector because the x t observations incorporate the effects of the
u t minus one shock. So there is no complete exogeneity and assumption A2 fails to hold.
Now, how about cross-section data? Would that make things better? Well, mostly not. Think,
for instance, of a sample of countries which you have and whose relationships you study.
We are in a connected world where countries trade with each other. So when one country is hit by a
shock, then economic variables in this one country will be affected by this shock.
And as a consequence of this shock, the export and import decisions of this country will be changed.
For the better or for the worse, it doesn't matter. There will be changes. There will be
effects of the shock which has hit a specific country on the exports and imports of goods
and services of this country. That's almost certain. But since this country trades with other
countries, this is the essence of the meaning that the country exports and imports, this other
country, which is a trading partner of the first country, will also be affected by the shock because
export and import demand of the first country will change. So clearly, other countries are also
affected by a shock which hits one specific country. And therefore, we do not have the fact
that the X matrix is completely exogenous to the shocks which have hit the economy,
even if perhaps the shock occurred in a previous period. It will nevertheless affect
the current period. And they're not only one country, but many countries which trade with
this one country. So assumption A2 is a very strong assumption which usually in economic data
you will find violated. There are some conditions, however, where assumption A2 may hold. For instance,
if you have regressors, which say, just illustrate the sex of persons, you have some type of
regression where you want to explain whether wages differ for people who are male or people
who are female. So there's wage equality for men and women. Then it may be that you have a
regressor matrix in which you just have a regressor or let's say two regressors which
indicate whether a person is male or female. So if it's a man, then you have a regressor
which has a one for each person which is male and a zero for each person which is female. And
if you have a woman, it's just the opposite. Obviously, these type of regressor matrices
where you just indicate the sex of certain people would be exogenous to the shocks because the shocks
can't change the sex of the people. Or if you speak of regions or countries which trade with each
other, then it may be that you have certain regressors which indicate which regions are
regions in mountain areas, for instance, or which are islands, remote islands perhaps, or which
share a certain border with another country with which it trades. So these kind of geographical
regressors would be regressors which are truly exogenous to the shocks because they cannot be
changed by the shocks. And in this case, it may be the case that you have a regressor matrix
where assumption A2 holds. But this happens only rarely that the complete regressor matrix is made
up of such regressors. In most cases with economic daters, you must at least check if there is not
some dependence between the U vector and the complete X matrix. Now, since it is such a
strong assumption, assumption A2, economists typically replace it by a weaker requirement,
and much weaker would be requirement of assumption A6.2, that it is only
the observation I which we have to look at. For each observation I, for each observation I,
we require that Xi and Ui are uncorrelated, which basically means that whoever decides on Xi does so
before she or she knows that Ui has materialized in a certain way. In that case, there cannot be
any correlation between Xi and Ui. And this is why we sometimes speak of Xi being predetermined.
Xi is determined before people know how Ui is determined. In such a case, that would be
assumption A6.1 already, so a little stronger. In such a case, this would also imply that we have
uncorrelatedness of the Xi and Ui variables, so assumption A6.2 would also be satisfied.
And we just need this uncorrelatedness between Xi and Ui. We do not require that Xi be uncorrelated
with Uj, or some j which differs from i, or that Xt be uncorrelated with Ut minus 1, which is very
unlikely for time series data. So assumption A6.2 that is very important to understand is much,
much weaker than assumption A2, and the same holds for assumption A6.1 or A6.3.
So in particular, when you work with time series data, you always have to check if shocks in t
do not correlate with the regressors of period t. The easiest specification is that typically when
you have a dependent variable of period t, try to explain it just by variables dated period t minus
1, so dated with a preceding period, but not with variables from the same period. However,
while this is easily said, it is not so easily done because most economic models would imply
that any dependent variable of period t is also affected by other variables which
were determined just in periods t and not in period t minus 1.
The contemporaneous correlation may also occur in other settings if we have
measurement errors in the explanatory variables. So even if we have a setup where the true variables
would have this property of being uncorrelated to the error terms, so we could think of when
you just look at your model where you could think of xi being uncorrelated with ui, then it's not
clear whether in the data this property is also fulfilled, because as you will see when we have
measurement errors in our variables, then we will also get this type of correlation. I'll show this
later to you. But anyway, as long as assumptions A7 and A8 are satisfied, we can establish then
that the plim of beta hat is equal to beta, because the plim of beta hat is equal to beta
plus the plim of this product of matrices here, and the plim of a product of matrices is equal to
the product of the plims of these matrices. So this is equal to sigma xx inverse c times 0,
and this is under assumptions A7 and A8 equal to 0, and therefore the plim of beta hat is equal
to beta, so beta hat would be consistent for beta if assumptions A7 and A8 are satisfied.
However, as I hope you have understood, beta hat is not unbiased anymore in finite samples,
so we do not have an unbiased estimator when we have a consistent estimator. These are different
concepts, unbiasedness being a concept in finite samples, and this can be true under strong
assumptions like A2, but if A2 is violated, then typically our beta hat, our beta hat here,
the least squares estimator, is beta plus this product here, and this product is typically non-zero
unless we have infinitely many observations that we can invoke asymptotic arguments.
I think now I have explained this at some level of detail, and actually for the second time since
I also explained it already on Tuesday. Are there any questions?
I don't see any, so I move on to an exercise where you may try to get a feeling of what
happens when you do this in MATLAB or any other programming language, so what you can do is similar
to exercises of this type which I gave you before. Generate in MATLAB again some regressor matrix x,
let's say with 100 observations and three regressors, just take random numbers as regressors,
and in this case you just compute the expression 1 over 100 times the sum over the 100 components xi
times xi prime. So you just compute this here and as a result you'll get a matrix which is the
average over the first 100 observations which you have. Now add another 100 observations to the
matrix x, so make x matrix of 200 by 3 without changing the first 100 observations. So leave
the first 100 observations unordered and add another 100 observations in terms of just random numbers
and again compute now the average over all those observations and continue like this to see whether
1 over n times xi xi prime converges to a finite and non-singular matrix. You can just do this for
for as many observations as you want and what you should check for is if this average matrix which
you compute here or there, if this changes by a great deal going from one number of observations
to the next number of observations and if these changes become smaller over time and you can
basically see that the matrix converges to some limit and then you can check if this limit which
you find there if this is actually a non-singular matrix and if you do this correctly you should
see that is indeed a non-singular matrix. So the assumption A7 which we basically test here or
simulate here seems to be a reasonable assumption. However there are some exceptions and one exception
is of particular importance. Suppose in your regressor matrix you have both a constant which
is normally the case and a linear trend in x which is very often the case. So your regressor
matrix x is made up of a first column which is just a column of ones for the constant and somewhere
you have a second column or the last column which is a linear trend so it's a column of entries one
two three four five and so forth up to observation and that would be the linear trend. If you have
this type of matrix then note that the x prime x matrix does not convert to a finite matrix
anymore right so there is an immediate violation of assumption A7 because x prime x as we know
is the sum over all the ends over xi xi prime. Now what is xi xi prime? xi for the first column
is equal to sorry xi for the first
no I should say for one of those entries when you compute xi times xi prime you will get a
multiplication of a row of ones with a column vector of ends so this thing here right which
means that you sum all the numbers from one to n so that is equal to the sum over all i's when i
runs from one to n and we know that this is equal to n times n plus one over two.
Now watch out that this element which is just one element of the matrix xi times xi prime
right this element converges to infinity right because already
or yeah it converts to infinity but but recall we multiply the whole thing by one over n
right so this n factor here cancels against the one over n but then we are still left with
n plus one over two so we see that one over n times x prime x contains an entry n plus one over
two and that converges to infinity so the matrix sigma x x is not finite in this case right so when
we have this set up that there is a constant and a linear trend in the regressor matrix
then our assumption a7 here is violated because the matrix sigma x x is not finite.
Now you may be worried about that because you think that happens rather often that you include
a constant and a linear trend actually you always typically include a constant and very often you
include a linear trend the good news about the bad news which i just communicated to you is that
actually the consistency property can be maintained or is maintained even if we have
a constant and a linear trend and the reason for this is the following when we have such a sigma
x x matrix where one element becomes infinite then this means that basically the inverse here
we compute the inverse of sigma x x converges to zero in at least one vector and that's fine
we don't have a problem with that because we want this product here to become to become zero
right we multiply by zero and multiplying is zero by zero doesn't cause any problem
problem actually it implies that the rate of convergence the speed of convergence
in the splint convergence property here is faster than usual so including a one over
including a constant and a linear trend leads to faster convergence to beta
and not too slower or no convergence to beta so we still have the property that the estimator is
consistent but you should be aware of the fact that this then uses different arguments than the
ones i have established here because we cannot make use of assumption a seven
assumption a seven is violated and then different techniques for proving consistency have to be used
but the result will be yes there's still the consistency of the least squares estimator even
if we have a constant and linear trend in our regressor matrix and the speed of convergence
is even higher than in the usual case so there is no problem but as i say the proof is different
more technical so i leave it out here
second thing which i already talked about x i and u i may easily be correlated macroeconomic
models so when x i and u i are correlated then we know that the expectation of the product x i times
u i is different from zero and then we have no consistency right and i asked you here to
demonstrate that this assumption a seven is violated for the estimation of the consumption
function um actually i think i miss spoke here or miswrote this exercise it should be assumption a
eight right demonstrate please that assumption a eight is violated for the estimation of the
consumption function in the kz cross model of equations 20 and 21 remember that i introduced
it to this model there or reintroduced you to this model there the the income expenditure
model which you've learned in macro well show that for the estimation of the consumption function
in this model you would not have consistency because the expectation of y i income and the
shock u i is equal to sigma square over u divided by one minus c and this is different from zero c
being the marginal propensity to consume so you see that here economics and econometrics meets
and you cannot not do econometrics by saying well i just assume certain things and and then i have
certain results but you always have to check whether in the specific model you work in or
in the specific data sets you work with the assumptions which i introduced you to here in
this lecture can actually be thought to hold and that is not necessarily the case in many instances
this is actually not the case and after some thought you will realize that those assumptions
which make life easy in econometrics are not satisfied and that we have to use econometric
techniques which are a little bit more advanced in particular instrumental variables techniques
to which i will introduce you later and where i will thank you many examples of why and how
these assumptions here may be a violated practice you have to use these type of techniques in order
to produce reasonable econometric estimates whereas if you just use the basic techniques
like the least squares estimator i talk about so much in this lecture now then every referee
will turn down your paper and we'll say well go back and study econometrics all right any questions
now let's turn to a violation of assumption a5 you recall assumption a5 was the assumption which
we added later to the Gauss Markov assumptions a1 to a4 we did not need a5 to establish the blue
property of the least squares estimator because a5 is the assumption on the norm normality of the
error term so the assumption on the error term following a Gaussian distribution be normally
distributed and this assumption we did not need for the blue property of the least squares estimator
in that sense the least squares estimator was as i said a non-parametric estimator or is a
non-parametric estimator but normality is a useful assumption to have when you want to test
when you want to apply standard tests like the t test or the f test or the chi-square test then
you always need a normal distribution and we made already an argument that you may get a normal
distribution asymptotically if the distribution finite samples of the error terms is not normal
so if assumption a5 does not hold there i will repeat this argument now with reference to our
model to our econometric model and therefore i first introduce a new convergence concept
for random variables because so far i have basically assumed plim convergence for random
variables once i also mentioned that there is a different convergence concept like convergence
in mean squares so that the mean squared error converges to zero that is even stronger than
plim convergence but here i introduce you to a concept for the convergence of random variables
which is even weaker than plim convergence namely convergence in distribution and the
definition of convergence and distribution says that a series of random variables xn
converges in distribution to a random variable x so if your random variable x would be the limit
distribution if it is true that the distribution function so the cdf the cumulative distribution
function of this series of random variables here converges to the cdf of the random variable x
and so if it is true that the limit of fn of x is equal to x for every point x at which f is
continuous now that's a technicality continuity here and then of course fn and f denote the cdfs
of xn and of x respectively as a shorthand notation what you typically read in the literature
and then what you what i'll use in this lecture here is that we write xn converges in distribution
that's what this little b here says to x meaning that the sequence of cdfs converges to the cdf of
capital x now as i already said plim convergence implies this convergence in distribution
so if a variable a sequence of variables converges in the plim sense to some limit
then it also converges in distribution but the reverse is unlikely not true so a convergence
in distribution is a weaker concept now suppose that our assumption a5 about the normality of
the error terms does not hold which is often the case in empirical work then you recall that we
had the central limit theorem which says that if a random variable epsilon i is iid
and satisfies the properties that its expectation is zero and its variance is constant as sigma
square then it is true that one over the square root of n times the sum of the epsilon i converges
in distribution to the distribution function of a standard normal no not standard now to a
normally distributed variable with expectation zero and variance sigma square and that as i
have already emphasized many times in this lecture is a very strong result because this holds for any
distribution of the epsilon i's right regardless of how the epsilon i's are distributed as long as
the epsilon i's are all iid so independently identically distributed we would know that one
over the square root of n times the sum over the n epsilon i's converges to a normally distributed
variable as the number of observations n grows to infinity i give you here as an exercise
a partial proof of the central limit theorem actually the part of the proof which is hard
to prove namely that is truly a normal distribution to which the distribution converges in distribution
this i do not let you show as an exercise that would be too hard but what i ask you to show are
the easy parts of the proof namely that the expectation of one over square root of n times
the sum of all the epsilon i's is equal to zero for all n regardless how many observations we have
and that the variance of one over square root of n times the sum of all the epsilon i's
is equal to sigma square for all n's so essentially what you are to prove is that the zero here is
true and the sigma square is true but what we are not to prove is that the distribution function is
normal distortion right so that's the hard part and the other parts a and b here are very easy
actually i i realized that some of you expect the exercises and the proofs i asked you for are much
more difficult than they actually are mostly the exercises the proofs i asked you for and the
exercises are easy proofs sometimes just one or two lines and so if you find an easy proof
then you can be quite confident that at least no greater level of difficulty is required i cannot
promise that it is correct but you you may be completely satisfied with the one or two line
proof in many cases of my exercises all right um now look at assumptions a7 and a8 again of which
we know that they are sufficient assumptions for the convergence in the plimps sense of the least
squares estimator as a plimps convergence implies convergence distribution we would also know that
these assumptions a7 and a8 are sufficient for convergence and distribution of the least squares
estimator so where exactly does least squares estimator beta hat converge to when the number
of observations increases you may possibly now say well it converges to a normal distribution
i want to show you that this is not the case right for very easy reason actually but be warned
that you have to be careful here look at the distribution of beta hat well we know
that under assumption a7 and a8 the asymptotic expectation of beta of beta hat is actually
beta because we know that the estimator beta hat has beta as its plimps i should actually change
the slide a little bit and not only write the symbol here which means that beta hat is distributed
as something which has this expectation and this variance but i should put a little a over this
tilde here to make clear that it is the asymptotic distribution it is not the finite sample
distribution you may misread this line actually as saying that it would be the finite sample
distribution that's wrong and this what i mean here by this tilde is the asymptotic distribution
and now that i read it again i see that perhaps it would be better if i put a little a over it to
indicate asymptotic distribution anyway the asymptotic value to which beta hat converges is
beta as we know and we also know asymptotically the variance would be sigma squared u times x prime x
inverse c now beta hat minus beta therefore is distributed as asymptotically distributed
as a variable with expectation zero and finite and asymptotic variance sigma square u times x
prime x inverse c as before so i just subtract beta on both sides of the sign here get beta hat minus
beta now what does this imply this implies that beta hat minus beta is distributed as a random
variable with expectation asymptotic expectation zero and then well i introduce those factors one
over n and one over n here where again please you remember that the one over n in those parentheses
taken inverse right is equal to n so n times one over n cancels and actually i haven't changed
anything in this notation as opposed to this notation here because the factor one over n
cancels against the factor one over n inverse c here but writing it this way you see that this
term here one over an x prime x inverse c by assumption a seven converges to a matrix sigma
x x inverse c and that's a finite matrix as we know right so essentially now we see here
that the variance is one over n sigma square u times a finite matrix now what happens with sigma
square u times a finite matrix when n goes to infinity obviously uh the matrix which is
the covariance matrix of beta hat minus beta becomes smaller and smaller as n increases
and when n goes to infinity the asymptotic covariance matrix here will be just zero
right so this whole thing converges to zero which means that we would have a degenerate
distribution actually no did normal distribution or anything like that but just a degenerate
distribution beta hat minus beta would asymptotically have an expectation of zero
and no variance anymore right so basically the least squares estimator shrinks to exactly the
true parameter beta with no variation anymore right so the distribution is asymptotically
degenerate does any of you have an idea how this relates to the result which i had previously
stated uh where did i stated um now perhaps i haven't even stated it in that form yet it is
still to come but um i had at least given you an idea of the result which i'm aiming at
that we have asymptotically a normal distribution for our least squares estimator what i have shown
you here is that the least squares estimator does not asymptotically follow a normal distribution
but rather shrinks to the true or the distribution shrinks to the true parameter value beta which is
completely in line with the fact that the plim of beta hat is equal to beta right because as you know
plim convergence says that the probability of beta hat being at least one epsilon apart from beta
that this probability goes to zero so beta hat actually becomes closer and closer to beta with
no variance at all asymptotically so where does now the normal distribution come in is there
anybody of you who recalls this from under graded econometrics or has an idea from what i said before
now volunteer well the the thing is actually trivial but i just want to draw your attention
to the fact the non-degenerate distribution which is then actually an asymptotically normal
distribution is only obtained if we scale the beta hat minus beta by a factor of square root of n
right so it is not beta hat minus beta which is normally distributed it is not beta hat which is
normally distributed but only square root of n times beta hat minus beta is normally distributed
by multiplying beta hat minus beta by square root of n we prevent
this term here from shrinking to a single point right when n increases
this term here actually increases too whereas beta hat shrinks to beta as we know
the product of the two then converges to a constant distribution and in particular to a
normal distribution right so we need the scaling factor square root of n to get another distribution
and when we replace the assumption a 6.2 by the slightly stronger assumption 6.3 so when we really
assume stochastic independence of xi and ui and not only that the variables be
contemporaneously uncorrelated then it is possible to show that under assumption a1
multi-two multi-two a3 a4 and a6.3 we have the asymptotic normality of the least squares
estimator in the sense that the square root of n times beta hat minus beta converges in distribution
to a normal distribution right with expectation zero and with covariance matrix sigma square u
times sigma xx inverse so that's the limit the plim of this matrix x prime x divided by n
inverse okay but you need the scaling factor square root of n and i mean this is actually
clear if you look at this at this expression here again because you see this term here converges to
something which is finite namely sigma xx inverse and the fact that beta hat shrinks to a degenerate
distribution at the point beta comes from the fact that in the variance term we have one over n here
right so when we primarily apply beta hat minus beta by square root of n then this increases the
standard deviation of beta hat minus beta so taking the square of this standard deviation
we would have to square the square root of n to get a factor n and this factor n in terms of
variances then exactly cancels against the factor one over n here leaving just as the variance sigma
square u times this fixed matrix sigma xx inverse c which you see here right so when we want to run
tests invoking asymptotic arguments we can make use of this result here but we always have to
pre-multiply our test statistic by a factor square root of n right typically the test statistic would
be constructed as beta hat minus the suspected vector of value of beta or the hypothesized
value of beta which we test under the null hypothesis so it can easily be that we test
a null hypothesis of beta being equal to zero right so beta hat minus the suspected value of beta
typically divided by the standard deviation and then pre-multiplied by the square root of n
so the practical relevance of this result perhaps in somewhat easier terms is the following
even if the shocks ui are not normally distributed we may use the normal distribution for inference
in large samples under assumptions a1, a3, a4 and a6.3 because then each regression coefficient beta
k hat which we have estimated is approximately normally distributed with an expected value of
beta k so of the true parameter and a standard error of sigma divided by the square root of n
times well the square root of c bar kk and c bar kk was the diagonal element of the sigma xx inverse
matrix. Therefore for sufficiently large number of observations n we can use the t-test and we can
use the f-test and even chi-square tests so all these tests which are based on the normal distribution
we can use if the number of observations is sufficiently large and then of course the question
is what exactly does this mean sufficiently large and there is no rule of thumb which I can
give you there but certainly not an exact mathematical result because it always depends
on how far is the distribution of the shocks ui from the normal distribution right if these shocks
ui were normally distributed well then we didn't even need to invoke any asymptotic arguments then
we can just use the t-st distribution as you know use f-tests and t-distribution is very close to
the normal distribution already for 25 or 30 observations so no problem if the ui's are
normally distributed if they are not normally distributed it may be that still their distribution
is quite similar to the normal distribution so it's quite close to the normal distribution
and it may well be that with say 30 40 50 observations you can already invoke asymptotic
arguments with very little error but if it is the case that the ui's are distributed
in a way which is very very far off the normal distribution then obviously it takes more
observations than just 30 40 50 to have a good approximation of the distribution of the regression
coefficients through the normal distribution there's no rule of thumb which may tell us when
we have sufficiently many observations who generate or to use the normal distribution
for testing purposes in order to give you some feeling for that i again ask you to
explore this issue in MATLAB or R or whatever you want to use so the setup is as in the previous
exercises and you can probably make use of parts of the code you've written for previous exercises
generate again let's say 100 by 3 matrix of regressors called x just use random numbers
for this and then again use a do loop which you run through let's say a thousand
times so in each loop you generate 100 by one random vector of shocks uj but now don't use
normally distributed shocks because then the issue is trivial and we have a normal distribution of
the irrigation coefficients but here this exercise i asked you to use uniformly distributed
disturbances right uniform distribution would mean on some interval let's say from negative
a to a you have the same probability constant density for each possible value of the error term
and that would be a uniform distribution and typically in MATLAB and in Gauss there are
commands which just generate you uniformly distributed disturbances right so and generate
the observations yj this whole vector the j refers to the loop which you run through a thousand times
so that's a vector y in loop j generated as x times some parameters and you may take them
as one one one but you can also use different values here class shocks which are uniformly
distributed okay so and then you have the regressor matrix and you have your
dependent variable which you observe then estimate beta that would be beta hat in loop j
beta hat j as x prime x inverse the x prime yj and save this estimate and do this a thousand times
and then take all the beta hat j's and plot a histogram for each component right you in each
beta hat j you have three estimates uh for this coefficient that coefficient and that coefficient
so for each of these coefficients plot a histogram and have a look at it whether the
histograms are approximately normal right if it looks like the central limit theorem already holds
for hundred observations if the arrows follow a uniform distribution rather than a normal
distribution how can you check that well either you just look at the shape of the the histogram
and see whether this has this kind of typical bell shape for the normal distribution or if you want
to be a little more exact then you can also check whether the third and fourth moment of the
distribution are close to those of a normal distribution by computing the schooness of the
distribution which should be zero if the distribution is is normal and by computing the
kurtosis of the fourth moment which should actually be three on that normal distribution and you will
have to see whether the kurtosis is approximately three for your three estimates here okay
and then of course you can do the same thing with uh higher numbers of n's so that would be a little
effort once you've made the first programming uh for the previous exercise it's very easy to just
increase n right and use n equal to two hundred five hundred thousand two thousand five thousand
whatever you want to see and observe how the histograms change as n increases right and
observe what happens to the schooness and to the kurtosis of the histograms if n increases
when you do this exercise you should see that the schooness approaches zero when you use the
when you use more and more observations and the kurtosis approaches three right and of course the
histograms should look more and more like the histogram of a normal distribution so i think
this would be a very useful exercise to get some understanding of what happens if the shocks are not
normal in this case uh uniformly distributed now the uniform distribution is in some kind of very
special distribution because if you have a normal if you have a uniform distribution distribution
which ranges let's say from negative a to plus a then you are sure that there will never be any
shock which is larger in size than a or smaller in size than negative a so the uniform distribution
really fixes you a certain range for your shocks and you know shocks extremer the negative a or
plus a will never occur this is a property which in some ways nice for the uniform distribution it
is also a little unrealistic we must say because we never have this knowledge in reality and even
under a normal distribution you could never fix any number saying well i'm sure that the shock will
not exceed this number in absolute value there will always be some probability that even more
extreme shocks occur than any given number so in that sense not this uniform distribution
is very far off the properties of a normal uh so you can redo now the previous two exercises
using a distribution for the error which in some sense is just the opposite of the uniform
distribution while the uniform distribution basically has no probability mass in the distant
tails of the distribution right more distant than negative a and a there's no probability mass for
any event anymore there are distributions which have considerably more probability mass in the
tails of the distribution than the normal distribution and one such distribution which
you may easily simulate in a computer is the so-called cushy distribution the cushy distribution
is just the t distribution with just one degree of freedom and any program like gauss metlab r
gives you commands to generate the t distribution and specify the degrees of freedom
so specify a t distribution normal students t distribution with just one degree of freedom
that's a very particular case actually of the t distribution called the cushy distribution
which is difficult because mathematically a random variable which is cushy distributed
doesn't even have an expected value technical reasons uh for that but the cushy distribution
doesn't have anywhere defined moments it doesn't have an expected value it doesn't have a variance
it doesn't have a schooness it doesn't have a kurtosis so it is indeed a distribution which
is very very different from the normal distribution and actually basically just with changing one
command hardly more than two or three keystrokes you can change the programs you have used in the
previous exercises to have the error terms generated by cushy distribution rather than by
a uniform distribution and then run the exercises again and see how fast the distribution of the
estimated regression coefficients beta hat converges to a normal distribution even though the
errors which generate your observations are coming from cushy distribution from t distribution with
just one degree of freedom okay i really recommend that you do those exercises um are there any
questions always worries me when there are no questions i hope you can follow what i've said
here um but i cannot do more than just offer you the possibility to pose questions
all right if not now uh you have another chance tomorrow morning tomorrow afternoon
let's turn to section two of this set of slides estimation problems so we have basically now
established the theory of least squares estimation both with finite sample properties and asymptotic
properties of the least squares estimator now let's talk about estimation problems and the
first problem i would like to discuss is multicollinearity you're probably aware of
multicollinearity my experience is that in undergraduates um uh professors seem to over
emphasize uh multicollinearity because when i speak to students in the bachelor's program
say they want to write their bachelor thesis with me and i ask them to do some estimation or when
they are in a seminar where they have to do some econometric estimation then they always come up
with multicollinearity and basically if they have any problems in their in their estimates then they
always first suspect the family multicollinearity actually this is usually not the case why
multicollinearity is a relevant problem it is typically by far not the most important
problem and multicollinearity typically typically can be dealt with really easily it is easy to
detect multicollinearity and it is easy to cure the problem and so mostly you should not have
problems with multicollinearity but of course you should be well equipped to detect multicollinearity
or approximate multicollinearity and to do what needs to be done if there is such a problem so
i will discuss multicollinearity here but i will also warn you that when you have certain problems
in your estimation if it is multicollinearity then you should really plan out about that easily and
solve it easily if it is not solved easily then typically it is not multicollinearity which is
the source of your problem okay so i use new notation here the regressor matrix x i now write
as x index one x index two up to index index k so x one up to x k small x's are just the columns
of our regressor matrix x previously i used the index i to denote the rows of the regressor
matrix x right when i had this sum of x i x i prime i talked about in today's lecture but now
these are the columns okay change the notation suppose that two of those columns are linearly
dependent so for instance x i is equal to alpha times x j obviously this is the most easy case
of multicollinearity which i now discuss with you um as you know linearly dependent can also mean
that x i is a weighted sum of more than just one column so we could also look at multicollinearity
as x i being equal to alpha one x j plus alpha two x k for instance plus
further alphas and further columns right so any kind of linear combination of other columns
being equal to one specific column like x i would be a case of multicollinearity but to save on
notation and to expose the problem which now resides most clearly i just use this completely
trivial form of multicollinearity where one column is just some constant times the second
column right i and j are of course different indices now our regression model is beta one
x one plus plus plus beta i column x i plus beta j column x j so a single out here the two columns
which are related by this multicollinearity problem problem here plus plus plus beta k
times x k plus vector u what i want to show you is this regression model here is completely
indistinguishable from the following regression model y is equal to beta one x one plus plus plus
nothing changes and now beta i plus one times x i so i just increase this regression coefficient
here by one i have beta i plus one here now plus beta j minus alpha times x j so
in order to compensate for the increase in this regression coefficient here i decrease
this regression coefficient beta j and transform it to beta j minus alpha x j
plus plus plus the rest is unchanged that obviously what have we done we have added zero
to the equation because i have added x i and i have subtracted negative alpha x j
since x i is the same thing as alpha x j this regression equation is equally true
as this regression equation here explains the observations equally well and then the problem
is of course for the least squares procedure which regression coefficients shall it identify
as the true regression coefficients as the best regression coefficients for the objective to
minimize the sum of least squares and the least squares procedure if you just think of it as a
person for for a moment right would say well the beta i and the beta j are just as good as the
regression coefficients beta i plus one and beta j minus alpha they just give exactly the same sum
of squares as the minimum right so i have two minima and actually when the when the least squares
procedure thinks about the problem a little longer it will find out there are not just two minima
which give exactly the same value of the sum of least squares but there are infinitely many
because also this model here is indistinguishable from this and that model this model here would
say y is equal to beta one x one plus plus plus nothing changed now beta i plus some lambda some
arbitrary lambda taken from the space of real numbers times x i plus beta j minus alpha times
lambda x j right so we can take any real number lambda here and increase this regression coefficient
by lambda and then decrease this regression coefficient by alpha times lambda right and
again we would have added just zero to the initial regression equation because lambda times x i is
equal to lambda times alpha times x j right so the regression equation has changed but the coefficients
are different of course from this coefficient here as long as lambda is different from one
right so that these two coefficients are also different and of course they are different from
beta i and beta j so essentially any regression coefficient can be chosen for beta i and we would
always find a regression coefficient for beta j which exactly offsets the beta i here in such a
way that the sum of the squares is minimum therefore what we say is that the regression
coefficients are just not identified right the least squares procedure is unable to find a unique
minimum for the sum of lists for the sum of the squared residuals and therefore it cannot
estimate beta i and beta j because infinitely many values of beta i and beta j
would be exactly minimal for the least squares problem
so that's the main message here in the case of exact multicollinearity the coefficients of x i
and x j are not identified note that it is only a problem for the coefficients of x i and x j in
this case because i have assumed that only x i and x j are connected to each other by this linear
relationship here all the other regression coefficients are identified there's no problem
estimating them but these here are not s i'm not i don't know so what would happen to the least
squares estimator if we just plug in some matrix x which has this multicollinearity problem that
x i is equal to alpha times x j or some more involved form of multicollinearity with more than
just one column of x being able to reproduce another column of x i taking in the form of a
weighted average now the problem is in this case here x i equal to alpha x j the x matrix does not
have full rank right the rank of x is equal to is smaller than k because we have the exact linear
uh the the exact linearity here between x i and x j and therefore when the rank of x is less than
k so when the matrix x doesn't have full column rank then the rank of x prime x is smaller than k
and therefore x prime x is not invertible and our least squares estimator just is not defined
right so when you try something like this when you set up a regressor matrix which has an exact
linearity and you try to estimate something your estimation routine will break down and the
estimator will return an error message saying that x prime x inversely is not invertible
right so you will not get any least squares result does this happen yes indeed it typically
happens for beginners of econometrics which make excuse me if i say it stupid mistakes right so
one thing which happens for for beginners sometimes is that they have dummy variables let's say for
male and female so for instance they want to check the wage of workers and test whether male workers
are receiving better pay than female workers for instance so for all the wages which they have as
dependent observations they have as regressors information on whether this is paid to a male
worker or to a female worker so they specify a dummy variable which takes on the value of one
if worker i is male and takes on variable zero is if worker i is female and then they say okay
this is a dummy variable for the male workers i also construct a dummy variable for the female
workers so d two i is equal to zero if the worker is male and is one if the worker is if the worker
is female actually so far this is not even a stupid mistake right you can you can do it that
way you can define the dummy variable for the male workers dummy variable for the female worker
but what you must not do is that you then use these two dummy variables joined with a constant
term your regressor matrix because if you did that if you had a constant so just a vector of ones
and you have one dummy variable for the male workers
and you have one dummy variable for the female workers then obviously there is an exact linear
dependency in your regressor matrix regardless of what other regressors you still have in this
matrix because it is the case that the constant x one is equal to the male dummy plus this female
right x one is equal to d one plus d two so these three columns here just have rank two
they do not have rank three right there's an exact linear relationship and this already
shows you what the cure of the problem is if you have an exact linear relationship
then the easy cure is you just delete one of those three regressors and it is completely
unimportant which of these three you throw out right you can throw out the constant you can throw
out the male dummy or you can throw out the female dummy it doesn't play a role just plays
a role for the interpretation of your regression coefficients but in terms of regression results
you can easily compute the regression coefficients which you would get for the
for any regressor you have deleted and the interpretation is always exactly the same
regardless of which regressor you delete so this type of problem is easily cured
note that in my example here i have not given you an example where x i is equal to alpha times x j
but i have already used somewhat more evolved example of x one being the linear combination
of two columns right linear combination with coefficient alpha one equal to one here and
alpha two equal to one here so it's slightly more complicated but it's in principle the same thing
right if you had this simple form of multiple linearity x i being equal to alpha times x j
well then throw out alpha times then throw out variable x j or throw out variable x i
you are just allowed to keep one variable in there but then everything is fine
okay so these kind of things happen typically with dummy variables male female or with seasonal
dummy variables right you get quarterly data you put in a dummy variable for the first quarter for
the second quarter for the third quarter and for the fourth quarter in principle you could do this
but then you're not allowed to put in a constant or what most people do put in a constant and a dummy
variable for the first quarter for the second quarter for the third quarter but not anymore
for the fourth quarter because the dummy variable for the fourth quarter would add up with the other
three dummy variables to the constant right so these type of errors sometimes occur with
beginners in econometrics but they are easily skewed now more common and more difficult is
the case of approximate multicollinearity so let's say x i is just approximately equal to alpha times
x j in this case when this is not an exact relationship here then the x matrix will still
have rank k but it will be the case that the x prime x matrix has determinant which is close to
zero you know a singular matrix has a determinant which is equal to zero so the x prime x matrix
in this case would be close to being not invertible so the determinant would be close to zero
what does this mean well the determinant is very important for determining the inverse of a matrix
we need x prime x inverse in our old sst matrix and recall from multi from your linear algebra
lectures that the inverse of a matrix is always the inverse of the determinant times some matrix
which is called the adjoint the adjoint is the matrix of cofactors and i'm not sure if you recall
this or if you've had this in linear algebra but if you haven't it's not so important right here
just remember this is some matrix which always exists the adjoint of x prime x exists even in
the case when x prime x is not invertible so it can always define it it's well defined the adjoint
and the inverse is always the inverse of the determinant times the adjoint the inverse of
the determinant does not exist if the determinant is zero because you cannot divide by zero
right so in this case x prime x inverse would be very large because if the determinant is close
to zero then we would have to take the inverse of the determinant here so we would divide something
by a number which is very close to zero so which gives a huge number here right so determinant of
x prime x inversely or one over the determinant of x prime x i should better say one over the
determinant of x prime x would be a huge number and that will be multiplied by some matrix which
is the adjoint of x prime x so we would get a huge covariance matrix x prime x inversely
now actually the estimate of the covariance matrix would be sigma hat square u times x prime x
inverse times this matrix here so the estimated standard errors of the regression coefficients
will be large and therefore the regression coefficients tend to be insignificant that's
a typical phenomenon of approximate multicollinearity that you see two or even more two regresses are
insignificant but when you eliminate just one of the two regresses then the remaining one suddenly
becomes becomes significant for two regressors again just experimenting on the on the specification
of the regression this problem is easily seen and easily cured well eliminate one of the regressors
right or at least eliminate the common component which a chance with another regressor which you
can done by some type of auxiliary regression for instance and then you should see that the other
regressor becomes significant if actually it is an important regressor and if multicollinearity was
at the source of the what was the root of the problem so that is easily found out by just
experimenting with the specification i have an exercise here for you suppose that y is equal to
beta one x one plus beta two x two plus u and for simplicity also the means are equal to zero and
all the variances are equal to one right in this case it is the case that the sample covariance
between x one and x two is equal to the sample coefficient of correlation r one two so these
assumptions here hold then you will get that the covariance between x one and x two speaking for
the sample is equal to the coefficient of correlation so basically covariance is equal
to coefficient of correlation so one over n x one prime x two or one over n x two prime x one is
equal to r one two now we have approximate multicollinearity of course when the correlation
is high or very low so either when it is close to plus one or close to negative one what you
shall show in this exercise here is that the estimated variance of beta one hat is equal to
the estimated variance of beta two hat namely sigma hat square u divided by n over divided by
one minus the coefficient of correlation squared and then of course when the squared coefficient
of correlation converges to one when it is close to one then you see of course that the variance
increases to close to infinity so you would get huge standard errors
and when you have to get a huge standard errors then of course your variables become insignificant.
You can do the same thing in MATLAB here is an exercise which will read out I've written down
everything I would just like to conclude today's lectures by giving you an example which we have
already studied and namely our earlier regression of stock prices on total effector productivity
and I consider in addition to total effector productivity a quality adjusted version of total
effector productivity such data exist I call it tfpq for quality adjusted and this is how the data
looks like you see the blue line is the regular tfp line which we had already used in this regression
and the red line is quality adjusted total factor productivity so you both have an upward
trend they don't look that similar actually and therefore we may think well why don't we test
them both in our regression here's the regression of stock prices on a constant on a linear trend
on total factor productivity on quality adjusted measure of total factor productivity and now you
see both tfp and tfpq are insignificant the t statistics are smaller than two and see the
standard errors just to remember this perhaps for the next outputs 2.6 here 3.8 there we don't know
whether this big or not but we will just compare important here is both variables are insignificant
now we may suspect that there is approximate collinearity due to tfp and tfpq being similar
variables and actually measuring similar things so let's eliminate one of the regressors here I
eliminate tfpq and suddenly as you see tfp is significant strongly indicating that we had a
multi collinearity problem and the standard error is much much smaller is 0.8 now before it was
2 point something right no it wasn't 2.6 2.6 so the standard error is much smaller now less than
one third of what it was before so that's also nice and if I kick out tfp and rather leave tfpq
in the regression then the standard error is much smaller again than the tfpq standard error before
which was 3 point something and again the variable is significant highly significant actually at the
one percent level it is significant so both variables taking as a single regressor in the
regression are significant but if we have them jointly in the regression then both of them are
not significant so that's a typical phenomenon of close or of approximate multi collinearity
right okay this concludes today's lectures are there any remaining questions
yes please
how much do the book versions from green differ from each other do you remain recommend to buy
the eighth version well no actually I mean I've said this already at the beginning of the lecture
they don't differ by much you can use old versions of those textbooks so there's no
problem with that the basic things which I explained here are just the same in each version
of the textbook so take the cheapest you would like to take take an older version that's all true
okay any other question
then thanks for your attention and see you tomorrow after