Welcome to today's lecture on estimation and inference in econmetrics.
We have concluded in the last week the second set of slides, you still see here the last
slide I presented on the usage of p-values.
I think that most of you this is familiar territory.
Let me nevertheless ask you, are there any questions which relate to this review of statistics?
If you have any, then please raise your hand.
I see at least one question.
Please type it in the chat.
So the question is, on the one hand, we say a random sample is drawn, but on the other
hand, we define it as a set of random variables.
Not sure I completely understand what is being asked, really not clear.
So perhaps let me just comment on that.
A random sample is indeed something which is drawn from the population.
So there's some explanation coming in, let me see, shouldn't the sample be a realization
of those random variables?
Yes, this is true.
The sample is the realization of those random variables.
So the sample is actually what we measure in terms of variables, what is the realization
of the random variables.
And you say that we define it as a set of random variables.
Probably the confusion here arises from the fact that of course you can look at a sample
also prior to its realization and then interpret it as a set of random variables.
So if we were linguistically absolutely exact, we should always distinguish between a random
variable prior to its realization, so a true random variable, and then the realization
of the random variable, which is actually not random variable anymore, but it's just
a real number.
And the same thing also applies to samples.
So usually when I speak of samples, I do indeed mean the realization of the random variables.
But it may well occur, I'm sorry for some confusion, some confusion arises there, that
sometimes I mean the sample prior to its realization and then indeed this is a set
of random variables.
I see that this may sound a little inexact and a little confusing at the outset, but
I hope you can get used to it and always infer from the context of what is meant.
In terms of notation, I think I'd distinguish very clearly between those two possibilities
in that I always denote the sample prior to its realization, the set of random variables
I should better say, which give rise to a sample.
This I denote by capital letters and the realized sample I denote by small letters.
So I hope this helps.
Any other questions?
A question, when will you release the student contributions result?
I cannot see the first week's results on Steene.
I haven't checked on that.
This is done by my secretary.
I thought that the student contributions results are already on Steene.
I will ask my secretary after the lecture why this is not yet the case, if it is true
that it isn't yet the case, but they should of course be published on Steene.
So perhaps you check again in two or three days and I'm sure they will be there then.
Okay, no further questions, then let me just go to one more kind of work incentive for
you.
Certainly, throughout this lecture, I will appeal to what I call your own research question,
which basically is just a thought experiment.
And the thought experiment goes like this.
I will now present you a list of possible topics for research.
And I will ask you to pick out one of those questions, not to conduct actual research
on this question, but rather to follow with this question in mind the topics of the contents
of this lecture.
Because every once in a while, I will ask you at the end of some material which I have
covered, well, how would you deal with this newly learned material in terms of your research
question?
What kind of research design do you have and how would you apply the methods or the contents
that I have taught you to this specific research question?
And therefore, you are asked to think about a real application with real world data of
the methods that I teach in this lecture.
So it is only a hypothetical research question.
You're not asked to do this research as a true econometric research, but as a hypothetical
question, you should decide on one of the questions which I now present you in the list
and apply the methods which I teach in this lecture throughout the lecture to this particular
research question of yours.
So I will every once in a while ask you about how this application would work in your particular
research question.
You do not have to bother about real world data and the availability of such data.
For the time being, you just have to choose a question and then I will confront you with
the first questions that I have to you when you apply the concepts of this review of statistics
to your research question.
So here's a long list of possible research questions like, for instance, what is the
effect of a minimum wage on unemployment?
Reasonable research question.
Lots of this research has already been done, but you could think about how you would address
this question in your own research and how you apply methods which you hopefully become
familiar with.
Or does access to electricity improve health?
Does it pay off for companies to advertise on Facebook?
Do tuition fees reduce the number of students?
Does course attendance improve your grade in this course or any other course?
Does increasing alcohol taxes reduce binge drinking?
Does playing a music instrument increase cognitive skills?
Does pay for performance increase stress levels of workers?
Do students' jobs decrease the performance of students at university?
Does increasing the generosity of paid sick leave improve the health of employees?
Does retirement make people happy?
Does unemployment make you sick?
Does graduation with honors improve your wages?
Does an internship increase the chance of finding a job immediately after graduation?
So long list of questions, just pick one of these, right, but then decide to have this
question throughout the course of this lecture and then apply the methods which I teach to
this question whenever I ask you to do this under the heading your research question.
So here's the first section, and if you have chosen a topic from the list of possible research
questions which I gave to you, then please answer the following questions.
For your particular research question, what would be the population of interest?
So this shall teach you or make you more familiar with the concept of a population that you
now think in terms of your particular research question.
What actually is the population of interest?
How would you draw a sample from this population?
Define some interesting hypothesis for your research question and state the null hypothesis
and the alternative hypothesis for the hypothesis which you want to research it.
What would a type one error look like in your case and what would be a type two error
given the hypothesis which you have formulated?
So it's just a thought experiment, as I said.
Don't bother to find real world data on your question, assume the availability of data
and then try to answer these questions as precisely as possible.
This is it for the review of statistics.
I will now move on to the review of basic econometrics, where I actually will start
with a really very basic stuff which all of you have already had.
But then I think I will move rather quickly to basic econometrics and matrix notation,
which is perhaps not so familiar to all of you but will be the basic method of how I
teach econometrics in many of the chapters you will have to go through in this lecture.
So please see where everything is still well known to you and where you sense that perhaps
you should invest some more time to familiarize you more with what I do here.
The beginning of this review of basic econometrics is certainly by content familiar to you even
though perhaps not by notation.
Later on perhaps there are also some aspects of basic econometrics which you may not have
covered in your undergraduate lectures.
Typically I think you should have had this covered but perhaps it's not proof for everybody.
So if you have deficiencies there then see that you closed.
I will spend quite a bit of time on the review of basic econometrics.
So I think I will take the whole month of November for this review of basic econometrics
and possibly even the first week of December we'll see.
But now it's November 1st and we start with it.
Here are two references which you may consult, are there many econometric textbooks around
and perhaps you have used other textbooks in your undergraduate studies then that's
also fine.
Typically they all teach the same type of content.
But if you want to use one of these two I would recommend either using William Green's
econometric analysis textbook or Jeffrey Woodridge's introductory econometrics where the letter
is easier to read.
William Green is certainly a little bit more advanced.
So if you want to have something which is closer to what you did probably in undergraduate
then the Woodridge text may be quite good for that.
If you want to come a little bit closer to what we do now on the master's level then
the green text may be the appropriate choice for you.
Okay, as I said we start really easy with a bivariate regression without any statistical
model yet.
So what we now do is just descriptive analysis.
So suppose we have some observations which we call y and we have other observations we
call x and we can match the observations y and x in such a way that we think we can describe
the observed values of y by some function of the observed values of x.
This is perhaps just an approximate description.
We don't really think that it is necessarily exact relationship which we have there but
somehow we have some hypothesis that y and x are related.
We don't see anything about causality so we don't know whether y causes x or x causes
y and these kinds of things.
We just say that the information included in x is similar to the information included
in y and we can retrieve the information in y approximately at least by just using
the data x assuming that there's some function f which retrieves this kind of information
on y from the data in x.
As always we try to make life as simple as possible and then make the first assumption
that f is a linear function.
So in this case we could write equation one as yi so the i-th observation on variable
y is equal to some constant b1 plus some constant b2 times the observation xi so the i-th observation
we have for variable x plus some approximation error which we call elac.
So this is why I can write an exact equality sign here now whereas I had approximate equality
in equation one because everything which was not included in the information we have on
x is now projected on this error term here which includes actually two different components
of an error namely first the component that well whatever the difference between the information
in x and the information in y maybe which gives rise to the fact that I have approximate
equality here only whatever is this difference this is in the ci here and then second there's
also the approximation part because when I assume that f is a linear function then obviously
I approximate possibly non-linear function f here by some linear function and this gives
rise to an approximation error so in as much as this approximation is inappropriate we
would have a second source of error in equation two and this is also included in the approximation
error ei here.
Of course this formulation already suggests that we can match observations from y one
for one with observations on x so we would have this index i which always tells us that
observation xi is matched to observation yi and observation xj is not matched to yi but
it would be matched to yj of course this also means that I assume we have the same number
of observations on variable y as we have on variable x but this is the usual setup.
Now let me give you an example for that data which I have uploaded to Steena please download
them you will get exercises for these data.
The example is taken from the pending broad table tables collection of macroeconomic data
which are adjusted in terms of purchasing power parity so PPP here for purchasing power
parity these are PPP adjusted data for basically all of the countries people could get data
from so in this case 187 countries we know there are a little bit more than 200 countries
in the world so it covers large size of the countries and I have picked here private consumption
data and GDP data where GDP data and private consumption data are not only real in the
sense that they are PPP adjusted but they are also taken on a per capita level and then
I chose a particular year in this case I chose 2007 so before all this mess with the financial
crisis and so forth studied and I will look at a cross section now over all of the countries
over all of the 187 countries so I have no time dimension here I just used year 2007
in order to study the relationship between income and consumption so the most basic relationship
which you have always encountered in upgraded studies first macro sessions a consumption
function basically so question is can we estimate a Keynesian type of world consumption function
where consumption would just be a function of current income so we have no permanent
income idea here just take the very simple Keynesian approach that currency and consumption
depends on just current income now as I said please download the data and compute the correlation
between private consumption and GDP you should get as a result that this correlation is rather high
namely 92.3 percent you see this here in the scatter plot of the data where you have GDP
and real per capita terms on the horizontal axis and you have private consumption and real per
capita terms on the vertical axis and very clearly there is a high degree of correlation between GDP
and consumption so between income and consumption now this is an example of the type of exercise we
are doing merely descriptive for the relationship between income and private consumption and one
idea one could now have is that why it seems clear that consumption can be written as some function
f of GDP it is also suggestive at least as a first approach to say this is perhaps a linear
relationship which we have here right and therefore we would use this linear approach which I have
shown on the previous slide and I would have to look for the coefficients b1 and b2 which I had
in this linear formulations obviously we can estimate b1 and b2 in many different forms and
such estimates can be either good or bad we are of course looking for best estimate we may possibly
have so what is the criterion for a good estimate as you know the criterion is that the error term
this ei which I had in my linear formulation should be rather small and therefore we use the
common approach of squaring the ei values and then looking at the mean squared error mean squared
error is one over n for n observations and is 187 in this case and I take the sum of the squared
errors which we have here which are as I say approximation errors or even substantial
errors so errors which relates to the fact that perhaps income doesn't explain all of
consumption all of this is in this ei here I square the ei's which are of course unobservable
currently but as a concept I can square them and then I take mean value values
very clearly n is a given number so it is not really important of whether we
we minimize one over n times the sum of the squared errors or whether we just minimize the sum
of the squared errors so we will do the latter and then this is the so-called least squares
problem which gives them also its name to the estimator based on the least squares principle
so I define a sum s which depends on the choice of the coefficients b1
and b2 in my simple linear hypothesis the sum s is the sum of the squared errors and I know the
error ei is should adjust my observation yi minus b1 minus b2 xi and of this expression I have to
take the square and then I sum all those squared here the least squares problem consists in choosing
b1 and b2 in such a form that the whole expression becomes minimal so this this notation minimize
over appropriate choice of b1 and b2 this expression here which is exactly the sum of the
squared error terms now obviously what we have to do when we want to minimize we have to take
the derivative with respect to b1 and with respect to b2 I do this here step by step first I look at
the derivative with respect to b1 setting this equal to zero so I get the first order condition
we get two first order conditions but this is the first first order condition the partial derivative
of s with respect to b1 so when I take the derivative then obviously I will have to
differentiate with respect to b1 so the squared term becomes a coefficient of two here
and the inner derivative is negative one which is this one here so the derivative as a whole is
the sum over two times yi minus b1 minus b2 xi times the inner derivative negative one
and this shall be equal to zero clearly since it is equal to zero the negative one here and the
two here don't really play a role so I can break the sum apart and move the negative components
to the right hand side of the equation so that I would get the sum over the yi's shall be equal
if I have an optimal choice of b1 and b2 shall be equal to the sum of the components b1 plus b2
times xi and now summing this b1 coefficient here over n units is of course nothing else but
n times b1 moreover the b2 is just a constant which I can factor out of the sum so the same
condition can be written as sum over the yi's is equal to n times b1 plus b2 times the sum over
all of the xi's and now dividing this equation here through by n I would get on the left hand
side of this equation here the mean of the yi's which is y bar so y bar shall be equal to b1
because the n goes away since we divide through by n plus b2 times the mean of all the x observations
of all the xi's here so this is our first order condition y bar shall be equal to b1 plus b2 times
x bar and the bars of course are just the sample averages then we take the derivative also with
respect to the other parameter namely with respect to b2 so same procedure we take the partial
derivative of s with respect to b2 then derivative in this case looks very similar to the first
derivative the only difference being that now the inner difference the the inner derivative is not
minus this is not negative one but negative xi because now we differentiate with respect to b2 xi
here so the inner derivative is negative xi when we take the derivative with respect to b2
okay and so in this case we can just divide the whole equation by two so who doesn't play
a road zero divided by two zero and we can again take it apart and would in this case
get that the sum over the products xi and yi shall be equal to the sum over b1 xi
plus the sum over b2 xi squared right the xi is multiplied into this first term here in the
parentheses and then we get negative xi yi this gives rise to this term if we move it over to
the other side and here i would have plus terms xi times b1 and xi times xi gives rise to xi
squared this then being multiplied by the coefficients we are looking for b2 and b1
dividing this equation here by n which is just a constant factor we would get one over n xi yi
is equal to b1 times x bar plus b2 times the mean of the squared values of the xis
here so one over n sum over xi squared now this expression looks still a little complicated with
these two sums in there but this can be written in a much easier way as you are probably aware
we'll see now in the following because we can exploit the fact that one over n times xi yi
which is exactly this expression here minus x bar y bar is actually an estimator of the covariance
between x and y then denote this x the estimator by covariance with a little hat to indicate that
is an estimator the same thing i can use for the term one over n sum over xi squared here which i
write down which i write here because i know that this term here minus x bar squared is an estimator
of the variance of x note that these estimators which i denote by variance with a little hat
and covariance with a little hat are biased estimators they are unbiased only asymptotically
but they are biased in finite samples because i divide here by n here or there but in order to
have an unbiased estimator i would actually need to divide by n minus one so they are not the best
estimators which we can have for the variance and the covariance but i will make use of these
type of estimators sometimes in this review of basic econometrics and as i say their notation
is then covariance hat or variance hat and for the unbiased estimator of the variance and the
covariance i will use different so just just bear it in your mind that covariance hat and
variance hat are the biased estimators of the of the covariance and of the variance of two
variables they are not completely bad estimators however they are actually the maximum likelihood
estimators of the covariance and the variance so they appear quite often in econometrics even though
we know in finite samples we can improve on these estimators by dividing through by n minus one
rather than by n okay and so why do i relate to these estimators of covariance and variance
simply because i want to simplify my second first order equation which currently takes
the form of equation five by getting rid of these ugly summation terms here and there and my
proposal to get rid of these terms is just to make use of these estimators here which however
are not exactly the same thing as the summation terms here because they differ by these terms
which are subtracted from them namely x bar y bar so the product of x bar y bar and the product
of x bar x bar squared we'll see how we can transform equation five such that these things
actually are included in equation five namely what we do is we go back to the first first
order equation which was equation four and multiply this first order equation just by x bar
right so the first first order equation the first order first first order condition
was y bar is equal to b1 plus b2 x bar and now i have multiplied the whole first first order
condition by x bar so that i get x bar times y bar is equal to b1 x bar plus b2 x bar squared
you see immediately that you could just divide through by x bar and then you retrieve the first
first order condition this equation six here which is an inflated first first order equation
i now subtract from equation five to get this expression here one over n x i y i
minus the term x bar x bar y bar here so i have on the left hand side of this equation
the estimator the maximum likelihood estimator actually of the covariance shall be equal to
b1 x bar plus one over n b2 times the sum of the squared x i bars minus b1 x bar
so minus the right hand side of equation six minus b1 x bar minus b2 x bar squared
and then you see the negative b1 x bar here and the positive b1 x bar here cancel precisely
and what remains on the right hand side of this equation is just the expression for the
maximum likelihood estimator of variance so it's variance hat of x so the second first order
condition using the first first order condition becomes that the estimator of the covariance
between x and y shall be equal to b2 which is the parameter we're looking for times the estimated
variance of of x so that's the second optimality condition we have essentially this condition here
and equation four so so we now equation four in this last condition for b1 and b2 we get
the optimal choices for b1 and b2 as b2 is equal to well the the the ratio between the
estimated covariance and the estimated variance of x right and observe this problematic factor
one over n which gave rise to the fact that the covariance hat is not unbiased in finite samples
or is not that the estimator of the variance is not unbiased in finite sample samples this
factor one over n appears here both in the numerator and in the denominator of this ratio
so it actually cancels so we could equally will use the unbiased estimator of covariance
and variance just doesn't play a role b2 is the ratio of the estimate of the covariance
and the estimate of the variance and from this we can easily retrieve what b1 is because b1 was y bar
minus b2 times x bar but b2 we have computed here so we just replace b2 by the ratio of covariance
hat divided by variance hat so this y bar minus covariance hat divided by variance hat times x bar
okay and perhaps it is noteworthy to see that this second first order condition
is a first order condition which relates the coefficient b2 to the second moments of x and y
so here is the variance of x right it's the second central moment of x and here's the covariance of x
and y which is also a second central moment of now the joint distribution of x and y so b2 is
something which tells us something about the second moments of the variables b1 is something
which is basically related to the first moments of the variance b1 is related to the mean y bar
which is an estimator of the expected value of y and it's related to x bar which is an estimator of
the expected value of x so b1 is related to first moments of the variables
using in order to determine the weight between y bar and x bar b2 is coefficient which is related
to the second modes of the variance but sort of as a as a general rule you can say the intercept
in a regression as the one we have just gone through is related to the first moment and
the other regression coefficient or the other regression coefficients
if we have many of them are related to the second modes of the variables
moreover the estimate b2 is similar to a correlation coefficient and on the next
slide you will get as an exercise to find out what is actually the dissimilarity why is this
not truly a correlation coefficient but just something similar to it yeah and the estimate b1
as i said the average of the observations y adjusted for the average of the explanatory variable x
here's the exercise i spoke of first exercise how does b2 differ from the correlation coefficient
between x and y if you don't see it directly then please go back to the formula which i have
given you in the previous lectures and on the second exercise please do the following
compute x bar y bar the variance of x and the covariance of x and y for this pampered
table data set of GDP and private consumption data which i've used in the example
um and then i ask you to compute the estimate b1 and b2 manually so by manually i mean of course
that you may use computer software but don't use sort of this fixed setup regression package like
the one which i will now use when i show you the os results but rather compute in some software
actually for this example it may still just be just be excel for for instance compute the mean
over the axis compute the mean over the y's easily done excel compute the variance and compute the
covariance let's say in excel and then use these data which you have in excel to compute estimates
the optimal estimates b1 and b2 to verify that what you have done say in excel use exactly the
same results as the ones which i will now show you using an econometric software maybe eius okay
so just to verify that eius is doing its job properly i will rather often in this lecture
give you exercises of this type where i write do something manually manually means always you can
use any kind of software but you should really compute estimates then from the moments of
the data rather than just running a regression uh routine so basically verifying that the
regression routines which are in your software do the right job later on you may find it perhaps
more convenient to use matlab or r or gauss or any kind of mathematical matrix oriented language
to do these kind of manual operations but for the time being you can still do it in excel and
actually many of these exercises you could also do in excel if you are not so familiar with r or
matlab or gauss i don't care what you use use any package which is familiar to you in a way if
you're comfortable doing these kind of calculations anyway when i use a commercial econometric
software like e-views i get this output here i will over the course of the semester rather often
present you output from e-views um output from other packages stata
uh matlab gauss or whatever you use is of course formatted in a different way
but mostly gives the same kind of information in particular on the highlighted coefficients here
you should get exactly the same type of result even if you do it in different software so the
e-views output is structured as follows we have a dependent variable which is private consumption
i denote it here at c c private uh we have 187 included observations mainly for 187 countries
in the year year 2007 and then we estimate that private consumption is explained by
some coefficient c some constant c which is just our b1 coefficient so this b1 coefficient would
be 1518 right and y is the symbol for gdp for income and so the coefficient here is actually
the b2 coefficient which would be estimated as 0.514 and you can always round after some digits
okay so these would be the two coefficients which we have estimated now as the latter one the ratio
between the covariance the estimated covariance between private consumption and gdp data why in
this case and here then the difference of the means mean of y adjusted sorry mean of private
consumption adjusted for the mean of gdp with the speed to coefficient in the way you have seen that
all the other statistics in this output i will explain later so just don't bother about them
you'll see this screen or a similar screen quite a few times in the following lectures
here is again the scatter plot of the data you've seen the data already nothing has changed there
i just put the regression line in the scatter plot the regression line in red is now
the linear function which i estimate for this regression here
now i'm not sure how much experience you have with econometric work in practice
um where you have some experience you should note that there's a problem here
does anybody see a problem in the scatter plot
please raise your hand if you do see a problem yes please uh i see two people uh raising their
hand please write in this in the chat what kind of problem you see first is already there
uh there's somebody there writing there's perhaps heteroskeleticity um yes no yes
probably there's also heteroskeleticity there uh what was the second suggestion
i haven't introduced heteroskeleticity yet but i'm sure you have encountered the term in
the undergraduate econometrics we'll come to this later heteroskeleticity was actually not the
problem i was aiming at when i asked you this question here there is even a more fundamental
problem and since the other participant does apparently not write his opinion yet
let me just explain what it is see when you look closely you see that for very low incomes
uh the data are almost all below the regression line so the errors which we make are all negative
for the low incomes whereas for the higher incomes most of the errors are above the regression line
there are a few also down here right but not so very many most of them are actually above
the regression line so it seems that the error itself still depends on income low incomes give
rise to negative errors and high incomes or middle incomes give rise to positive errors mostly at
least right so this suggests that we have not explained everything yet in private consumption
which we may be able to explain by income and i will show you a different graph which reviews
this problem much more strikingly than it is done in the scatter plot here namely when i plot
the estimated residuals ei against the logarithm of GDP then it looks like this
see that's an eview's graph which you can produce you have here the residuals this is the ei
here is the zero line obviously the eis are centered around the zero line but we see that
for low incomes and i measure you come now in logs in order to make the phenomenon
uh to come out of the data a little bit more clearly um you see for low incomes almost all
of the residuals are negative up to here approximately right and for higher incomes
many of them at least are positive not all of them as you see there are also quite a few which are
negative but on balance you could verify that many more are positive than there are negative
errors in here the answer given by one of you that there's also evidence of
heteroskeleticity was also correct because you see that the size of the error seems to depend
on the size of of income on the magnitude of income so with smaller incomes the errors are
the absolute value smaller than when income is great so that is also evidence of heteroskeleticity
this right and we probably deal with it um by uh also dealing with this other problem which i
pointed out namely that the errors for the low incomes are negative all of them whereas the
errors for the middle and higher incomes are mostly positive so and perhaps we should go back
to um this uh thing here again to this graph here again because uh this will already hint a
little bit what kind of modification uh we may take because actually when you look at the data
you may say perhaps the idea of taking linear approximation through the data was not the best
idea at all perhaps actually uh this scatter plot doesn't really suggest something linear
but rather suggests something with a little curvature perhaps it's even easier to see when
you don't have this misleading linear uh uh regression line in there but you rather look
at this uh thing here but if you really had a linear relationship starting out here with
the slope suggested by the lowest income observations then actually linear line will
always almost move like my little hand here can't do it quite exactly but after some time you would
think well linear line is perhaps a little bit off the line is more going that way rather than
describing good fit for the data here for which perhaps you're better advised to take something
which has curvature so some type of concave relationship between GDP and private consumption
in the cross section could be appropriate here and this is exactly what we'll do now
we seem to have systematic error right poor countries consume less than linear and linear
consumption function would suggest so we try a concave uh specification for our function f
by uh taking the logarithms of the data so now i denote by lny the log of GDP and by lnc private
the log of private consumption and again my regression is actually a linearly specified
regression but now the logarithms of the data so um the fact that data suggests some non-linear
relationship that does not necessarily imply that we have to use a non-linear function f
it may suffice and actually many cases it does suffice to non-linearly transform the data
and use the non-linearly transformed data in a linear regression approach so in a linear
function like this one here so the approach is now non-linear in the data but uh the relationship
between uh variables uh the transformed variables is still a linear relationship log of private
consumption is equal to b1 plus b2 times log of private income plus some regression error ei
now obviously the b1 and the b2 and the ei here denote now something completely different
from what the same symbols have denoted in the first approach in the linear approach just for
convenience i still use the same notation right um but but please bear in mind this b1 which we
estimate now is of course something not only numerically but also conceptually very different
from the b1 we have uh dealt with in the previous approach and the same how it's broken too and the
eis are of course of course and also uh different i've noted this down here we just use the same
symbols but they mean something different okay here is uh the regression result and now you see
we have again a constant which takes a completely different value from what we have before and here
so this is the estimate of b1 and here we have the estimate of b2 which describes us uh how the log
of gp translates into uh private consumption or better into the log of private consumption
i have highlighted now a different part of the output here namely this part where uh evius writes
sum of squared resilience right so this is the value of our objective function we wanted to minimize
our objective function so this is the lowest value which evius could achieve in terms of the
squared resilience right so it's this sum here uh the sum over all of the ei squared which is given
in the output there but just looking at this one output um this value is of no informative uh
content uh we can't really use it only if we run different specifications on the same set of data
so on the same logarithm data then we could compare uh the sum of squared resilience with other
sums of squared resilience in other approaches you can see where we achieve the minimum value
there's no point of course in comparing the sum of squared resilience here which is basically 20.3
with the sum of squared resilience which we had here which was a huge number as you see because
here we have completely different units in which we worked so this has nothing to do with each other
because first they never not logarithms but rather PPP adjusted dollar now they are unlocked so these
cannot be compared but if we have a second specification in terms of logs then we could
actually compare uh the sum of squared resilience with the other sum of squared resilience we would
obtain the second um approach all right exercise for you also reproduce in
some software the value of the sum of squared resilience
if i now plot the data uh so another scatter plot i use the logs of the data so this is the log
of GDP this is the log of private consumption here are my data and you see that the regression
line seems to be quite good in the sense that it really looks like this is a linear relationship
true we still have let's say one two three four negative observations at the start of the sample
for variable incomes but there are also already two observations exactly on the straight line and
then at the latest here we have quite a few positive observations so this can well be a random
event that the first it's actually just the first observation was negative and the residual was uh
then the residual will already be zero uh here the residual is negative negative negative zero again
then positive negative positive negative so that can well be uh the result of some random event
that we have kind of sequence of negative and positive resilience suggested by the graph
here we would probably not take this as evidence uh for some kind of further hidden non-linearity
but rather seems that this regression explains uh the data rather well and um now coming back
to this one answer which was supplied to my question the evidence of heteroskeleticity has
also uh finished right now it doesn't look like the size of the errors for low incomes
is smaller than the size of the errors for high uh incomes so basically probably by taking the
logs of the data we have also dealt with the problem of heteroskeleticity
so it seems quite uh satisfactory um still one should have a look at the distribution of the
errors and the distribution of the estimated ei terms and i do this here by looking at a plot of
the histogram of the residuals and now you see well the histogram is like these light blue bars
here i can also estimate with certain methods um which we will discuss in this lecture but
just to mention it here because if this provides this possibility i can estimate a density
function pdf for the residuals and when you look at this either at the blue bars or at the red
kernel estimate of the pdf then you see that this is certainly not a normal pdf so it doesn't seem
that the residuals have a normal distribution now i i told you that there is no necessity
in empirical work that the residuals should have normal distribution this distribution here is
clearly skewed to the left right um and this is fine i mean this this may occur uh there there is
as i say no law which requires the residuals to have a normal distribution still often it is
advisable to study this issue and see well have we really explained everything in the dependent
observations um or can we perhaps improve on this estimate here we won't go down further
in this route because this would require quite a bit of more empirical exercises here but leave
it at this because there's nothing wrong with a non-normal residual distribution
rather we move on to multivariate regression so far we just had a binary regression one
explanatory variable and uh independent variable and now we suppose that we have one dependent
variable which is doing not apply and we assume that there is more than one explanatory variable
x1 to xk where i already use this kind of kind of terminology calling the x's here explanatory
variables which in some sense may already suggest that i interpret them as causal variables which
explain why but it is not meant this way it's just the usual language to speak of the exercise
explanatory variables um but i do not want to suggest that this is already causality which we
establish here only change is we have more than one x so we have k axis the k can of course still be
just equal to one but what is interesting to us now is k greater than or equal to one so
actually what is interesting is k greater than one so y is approximately some function f of x1 x2
up to xk an example may for instance be that y is measured to wages or burgers and you may explain
the observed wages by different factors for instance by education or by experience or by
the age of the worker or by the sex of the worker whatever so there may be different explanatory
variables for the y now suppose again we have n observations on the workers for instance right
so y i um is the observation on one burger and we have n such observations
in this case for the explanatory variables we would use two indices x i one is the first
explanatory variable for the i th burger right and x i k is the kth explanatory variable
for the i th burger let's say variable one is education and variable k is sex and in the green
we also have and the variable for experience and education and possibly other variables
so our regression would be of the form y i is equal to b one times one so just providing for
constant here b one is not a coefficient of any of the observed axes um b two times x i two
b three times x i three plus plus plus b k times x i k but actually i have now um i have now not
used um an x i one here so this one you have to think of as the x i one in this case x i one would
just be a constant and x i two for instance would be education all right it is very convenient to have
a matrix notation for this type of data if we just speak about the observations we have for verca i
we would use a vector x i for all the variables which characterize
possible uh reasons why uh verca i has a certain uh wage so in this case x i would be a vector one
x i two x i three x i k so this would be a vector which always has some component x i in it and then
the column the second index would indicate which variable it is it is a vector of individual
characteristics in this example setting of explaining wages for wage of uh verca number i
and we would similarly have a vector of unknown parameters b one up to b k and we denote this
vector by b so observation i can then be described as y i let's say the wage of verca i is equal to
x i prime times b plus the error epsilon i which is a scalar right so we would take the transpose
here of vector x i i denote transposes by primes i would take the transpose of x i prime and then
multiply by b you see this vector here has k rows so x i prime would have k columns this would match
well with the fact that the vector b has k rows so x i prime times b is just again again right
that's uh yeah the the matrix or the vector product of a one by k vector with a k by one
vector so it is of dimension one by one this is however just for observation i and we have
n observations so what we may do is that we collect all the wages which we have in one
vector y then we have y one y two y three up to the wage of the nth uh verca and then we have
all the verca's characteristics which are now right as x one prime for the first verca x two
prime for the second verca and finally x n prime for the last the nth verca all these uh
row vectors x one prime x two prime x n prime are multiplied by a column vector b of unknown
coefficients and then i collect all the error terms in a vector n by one vector of error terms
e one up to en and i denote this vector here by just e so more compactly i can write this as y is
equal to x b plus e and x is this matrix here because this is truly a matrix with n rows and k
columns whereas y is a n by one column vector and e is an n by one column vector
x is called the regressor matrix and it is made up of a column of ones which by convention is
typically the first column and then come all the other explanatory variables ordered by variable
so the columns they are all the different variables like education experience age
sex and the rows they are all the verca's um so the second row would give us the characteristics
of the second verca the sum of the squared residues in matrix notation would be e prime e
this is just the same thing as yeah some of the squared residues right e prime e is the
inner product of the vector e so that's some of the squared residuals
please verify the following identity as an exercise if we take this matrix x in the way
in which i have to find it then x prime x is the same thing as the sum over all ends of the small
x's x i x i prime and x prime y is the same thing as the sum over all ends over x i y i
this x i y i actually well defined yes indeed it is x i is an n by one vector and this is just
multiplied by a column just by multiplied by a scalar excuse me just multiplied by a scalar
so a scalar we can always multiply with a column vector right and x prime e so the product of the
regressor matrix and the vector of of errors is just the same thing as the sum over all the x i's
which are n by one column vectors um times the e i
so and the ei is again a scalar okay um so we'll use these identities sometimes so please make
it clear that you understood that this is true and after you've understood this so basically
gone through it through it in your mind then verify the first equations in some software
MATLAB or R or whatever and using any kind of data for x and y obviously this third equation
you cannot verify because you don't have any data on e typically you don't know what the true errors
are so we will now write the least squares problem as a problem written in matrix notation
so our problem is to find the vector b such that the vector product e prime e is minimal
so the same type of problem is the one we had written in scalars before is now written as
minimize over choice of vector b e prime e and what is e prime e where we know e the vector of
um errors is the same thing as y minus x b i'm just using this
simple equation here e is equal to y minus x b so substituting for e y minus x b
or for e prime y minus x b prime i get this type of product um you know probably how the prime
uh is translating to matrix products we would in this case have the same thing as y prime
minus b prime times x prime because the order of multiplication is reversed when you take the
transpose of a matrix product of x b the parenthesis prime is the same thing as b prime x prime
times y minus x b and in this case you can then just multiply out component wise so you would get
y prime times y is y prime y and y prime times x b with a negative sign is negative y prime x b
and b prime x prime y with a negative sign is negative b prime x prime y and finally negative
b prime x prime times negative x b is plus b prime x prime x b it's the usual arithmetic
of equations which you see here in vector four now there's one thing which is important to note
because you have y prime x b here and b prime x prime y here both with a negative sign
at first sight it seems these are different expressions right but in fact they are not
why are they not different expressions we first note that this expression here is exactly the
transpose of this expression here or the the opposite way opposite way this expression here
is the transpose of this expression here right taking the transpose of this thing here would
mean that you have to reverse the order of the vectors so start with y it would be y prime times
x times b which is exactly the thing here y prime times x times b so these two are transposes
transposes to each other and moreover y prime x b is a scalar so this is just a one by one matrix
right because y prime is an n by sorry y is an n by one vector so y prime is one by n in terms
of dimension one by n b multiplied by multiplied by the x matrix which is n by k gives a one by k
matrix the one by k matrix multiplied with b which is a k by one vector gives me a one by one
result so this is one by one and it is the transpose of this thing here which is also one by one
obviously the two things are the same the same because a scalar a one by one matrix taking it
as transpose doesn't change anything right so we can also write this as y prime y
this thing here minus two times b prime x prime y plus then the fourth term b prime x prime x b
and we shall minimize we want to minimize this thing here by appropriate choice of b
obviously minimizing this expression here is done only with respect to the second term
and the third term because these terms depend on the choice of b
right we have it here there y prime y is completely independent of b so we can just
throw it away we just have to minimize negative two b prime x prime y plus b prime x prime x b
how is this done well minimizing is always means always taking a derivative there are
rules for matrix differentiation which i give you here without proof you can actually verify
this just in your mind that this is true but you can also learn this by heart if you like
if we take the derivative of an expression like b prime and some matrix this case x prime y
b prime times some matrix and then taking the derivative with respect to b is just equal to
this matrix x prime b so this is like usual linear derivatives or taking the derivative
of a linear expression the b prime just vanishes and the matrix in this case x prime y is the
derivative we take the derivative of a quadratic equation where we have a quadratic expression
excuse me a quadratic expression where we have b prime x prime x b with respect to b
then the derivative is two times x prime x so essentially it's always the b prime which
vanishes here right and it vanishes here and in the case of a quadratic equation you then also
have this usual term of two as a factor by which the remaining matrix x prime x b is multiplied
so if we take the derivative of this expression here with respect to b right
we take this derivative here then the first order condition is minus two times x prime y
plus two times x prime x b shall be equal to zero obviously we can divide by two
and we can move the negative term to the right hand side of the equation then we are left with
x prime x b shall be equal to x prime y this is a whole set of equation actually these are k
equations for k unknowns namely the k components of the k-dimensional vector b these equations
the set of equations is called the normal equations and it's very obvious when we want to
derive what b is we have that we have to solve the normal equations for b which is easily done
if it is true that the matrix product x prime x is non-singular if we can compute the inverse
of x prime x so if x prime x inversely exists then we know that b is equal to x prime x inversely
times x prime y and this is the least squares operator estimator right this is the least squares
estimator which we have derived here in matrix notation is it trivial that x prime x inversely
exists do we have any reason to believe that the matrix x prime x is not singular yes indeed we do
this is typically the case i just asked you to verify this in a very simple MATLAB or any other
kind of software exercise where you compute b manually oh no sorry this is not yet this exercise
but but just use MATLAB to compute the b manually as x prime x inversely times x prime y
for the sample data set which i provided to you where you have here the log of
private consumption and you regress this on a constant and on the log of gdp and then again
compare with our earlier evious output and if you have done this exercise correctly then you should
get the same results as those which we had in our evious output now here comes the question
is it true that x prime x inversely exists suppose that the x matrix has n rows as we
have always said and take columns and that n is greater okay so we have more workers which we
study than we have explanatory variables that should usually be the case in regression analysis
well if n is greater than k then x is a rectangular matrix and therefore it is certainly singular
however this doesn't matter for the question of existence of x prime x inversely what doesn't
matter for the existence of x prime x inversely is the question of whether x has full column rank
so if it is true that the rank of x is k
k being smaller than n but at least the rank must be k because if the rank of x is k so if the columns
of matrix x are all linearly independent then it is true that the rank of x prime x is equal to k
and if the rank of x prime x is equal to k then x prime x prime x inversely exists because
x prime x is a k by k matrix right and so the fact that the rank of x prime x is equal to k
would then mean that k has full rank but then x prime x has full rank k right so if x has full
column rank if the rank of x is equal to k then x prime x inversely exists and that is almost
always the case unless you have an exact linear dependency among your regresses which means that
you have a superfluous regressor in your regressor matrix which you can easily
kick out because you know all the information this regressor is contained in some of the other
regressors in just the same way to convince you of this fact perhaps the best thing is that
in your software be it MATLAB or whatever just generate random matrices of size n by k with n
greater than k call these matrices x right the content of these matrices are just random
numbers all these programs have random number generators so it's not easy not difficult
very easy actually to generate random matrices x of size n by k and then we just take random numbers
verify that x prime x has always rank k so the inverse of x prime x always exists
right i mean i offer you bet that you won't be able to find any x prime x matrix drawn from
random numbers right which would have rank lower than k so it's always the case if they are random
numbers that you will have rank k for x prime x now one important thing is that the inverse of
x of course does not exist because as i said x is singular there is however the generalization of
the concept of a an inverse matrix um this generalization of a concept of an inverse matrix
is the so-called pseudo inverse which is denoted by a plus for some given matrix a it is actually
the case that every matrix or also the the non-singular matrices really every matrix a
has a so-called pseudo inverse which is denoted by a plus and this pseudo inverse is also called
the Moore-Penrose inverse after its two discoverers Moore and Penrose. Moore discovered this
inverse already in the 1920s but nobody understood his publication and was forgotten and Penrose
then discovered it again in the 1950s unaware of the work of Moore and developed its properties
and only after Penrose has published his papers other people came up and said hey this looks like
what Moore has published years ago so that's why it's called the the Moore-Penrose inverse.
What exactly is a Moore-Penrose or pseudo inverse for a matrix x for a matrix a?
So given any matrix a there is just one matrix a plus this is why I write it is a unique matrix
there's just one matrix a plus and there always exists such a matrix a plus which satisfies four
conditions which I have listed here. One condition is that a is the same thing as a times a plus
times a right and if you just want to compare this with the regular inverse if a plus were the
regular inverse then clearly a times a inverse is just the identity matrix so this we can forget
and then a is equal to a all right the regular inverse also satisfies all these four properties
which you actually shall show in the exercise which is down here as the first task in this
exercise all right so four properties one property a is equal to a times a plus a
second property a plus is equal to a plus times a times a plus
third property a plus times a is symmetric so a plus times a is the same thing as a plus times a
transposed and fourth property a times a plus is also symmetric so it's the same thing as a times a
plus transpose note that a plus a is not the same as a a plus right the order of multiplication there
plays a role in the case of a plus b the regular inverse of course this would be the same thing
but for the generalized inverse the pseudo inverse this is not the same thing so there for any matrix
a there exists one and only one matrix a plus which satisfies all these equations here and
actually you can solve systems of linear equations using the pseudo inverse which in many cases then
involves using some type of solution a plus times a so the a plus matrix really functions
in a very similar sense in a very similar way as the regular inverse but i won't go into
the general theory of solving linear equations but the Moore-Penrose inverse is important because
it comes up very naturally in the derivation of the least squares estimator so to better understand
the Moore-Penrose inverse please go through this exercise here the first part of it i have already
mentioned then we show that a plus is equal to the inverse of a if a is regular so if a has full
rank and therefore is non-singular and then second show that the pseudo inverse of some matrix x
is equal to x prime x inverse c times x prime if the matrix x has full column rank so basically
if x prime x inverse c exists right in this case x plus is the same thing as x prime x inverse c
x prime and obviously x prime x inverse x prime is exactly the expression we had here in our
derivation of the least squares estimator there's x prime x inverse c times x prime so this is the
Moore-Penrose inverse of x and this is being multiplied then by the vector of observations
y to give the optimal estimates for the b vector all right another MATLAB-based exercise
generate again random matrices of size n by k and verify numerically that x prime x inverse c times
x prime satisfies all the properties one to four which define a Moore-Penrose inverse so verify
that x prime x inverse c times x prime is actually x plus
as I already said the least squares estimator is simply b written as x plus times y there's
a very simple intuition for that if it were the case that n is equal to k so we have as many
observations as we have explanatory variables and if the x matrix were regular and regular is the
same thing as non-singular and extra regular then we could just set b equal to x inverse c times y
right so all the errors e would in this case be zero but in the usual case in our usual applications
we have many more observations hopefully then we have professors so if n is greater than k
we simply replace this x inverse c here in this thought experiment
the squares estimator we simply replace the x inverse here by the Moore-Penrose
inverse and therefore we have a generalization of the concept of an inverse matrix here
now how would we write the resilience after we have derived the squares estimator we know the
e whose sum of squares we have minimized the e is just the difference between y and x b
initially we did not know what b would be but now we have an estimate for b which is the least
squares estimate so we can compute y minus x b b is x plus times y so this is y minus x x plus
times y right x plus y is the least squares estimate okay in this case in this case it's
still the estimator y minus x x plus y is equal to e so obviously we can factor out the y vector
here and write this as identity matrix i minus x x plus in parentheses times y
this matrix here which appears often in econometrics identity matrix denoted by i minus x x plus
i call matrix m so i can write e is equal to m times y now important note three important
properties of this matrix m here first thing is that x prime m is equal to zero so this m matrix
and the regressor matrix are orthogonal to each other why is x prime m equal to zero
well very easy if you multiply from the left side x prime to this matrix parenthesis here
to this m matrix here you would have x prime times identity matrix is x prime so x prime
minus x prime x times x plus but x plus is x prime x inverse e times x prime
so we would have x prime x times x prime x inverse e which cancels times x prime
so we would have x prime minus x prime which is zero right so x prime m is equal to zero
and the same thing holds when you multiply x from the other side from the right hand side
to matrix m so m times x is also equal to zero as you can very clearly convince yourself
and the third property is that m is a so-called idempotent matrix namely it is the case that m
times m is equal to m you can also easily verify that by just multiplying out i minus x x plus times
i minus x x plus you'll see that i minus x x plus times i minus x x plus is equal to i
minus x x plus so m by m i m times m is equal to m this is the property we call idempotent matrix
m is idempotent regardless how many times we multiply m by itself and the result is always m
as exercises before we conclude today's lecture show that these properties which i just
have introduced you to imply that x prime e is equal to zero so the regressor matrix
is orthogonal to the errors which we have estimated to the regression resilience
further show that the sum of the ei's of the regression residuals is equal to zero
if the regression contains a constant term that's an important issue if the regression
contains a constant term so if one of the columns of x is constant just a vector of ones
then the sum of all the errors will be equal to zero
well and again in MATLAB or R or whatever matrix language you like to use verify the
properties a and b for our sample regression of the log of private consumption of the log of
gdp and a constant so verify that a and b will not hold if you do not include a constant in
the regression this is part of of problem c in this exercise
that's it for today are there any questions
raise your hand if there is a question otherwise you have the opportunity of
raising a question tomorrow when i start the lecture in the afternoon
i assume that most of what i did in the first part of this lecture was well known to you and
perhaps the second part was not quite as well known in particular if you haven't had
matrix notation for econometrics so far in this case please work well through what i have presented
to you today because i will work matrix notation for a long time now and you should be well familiar
with it i do not see any questions so thank you very much for your attention and until tomorrow
i have to