We will talk today about hypothesis testing, or at least this is with which we do at the
start of this lecture, I move through hypothesis testing rather quickly because we have done
this in principle already in slide two of this lecture.
I just would like to remind you that we have our estimation, our regression, which we
talk about, here is a variant of it, the regression of the log of private consumption on the log
of income with cross-section data from various countries, and that I am in the process of
explaining to you the meaning of the regression output, which we get from regression packages
like eViews, which I take here as an example, other econometrics programs give similar output,
and I want you to understand what this means.
We have covered already a number of the statistics that you see, for instance, on the slides
here, like for instance, the coefficient estimates and the standard errors, you should
now know how they are computed, where they come from.
We've talked last lecture about the R-squared and the adjusted R-squared, you know what
the standard error of the regression is, and you know, of course, what the sum of squared
residuals is, or these statistics like the mean of the dependent variable and standard
deviation of the dependent variable.
We will now move to basically this block here, t-statistics and probabilities, when we talk
about hypothesis testing, which is section 1.4, I think it should be 3.4, but never mind,
and so the fourth subsection in slides 3.
Now, as you will recall, we have so far not made any distributional assumption, rather
we proved the blue property of the least squares estimator, that is to say, we proved that
the least squares estimator is the best in the sense of variance minimal estimator in
the class of linear and unbiased estimators of our parameter beta.
On this property, the blue property is valid for any distribution of the error, so we do
not need to assume a particular assumption, a particular distribution, like for instance,
that the errors are normally distributed.
It is a property which is valid regardless of what distributions the errors follow, and
therefore it's a very strong property, actually, of the least squares estimator.
Now, in terms of hypothesis testing, we need more than that.
We need to make an assumption about the distribution of the errors, and this is what we do in
assumption A5, which you see in the green box here, and now assume that all the errors
ui are independent of each other, and that they are normally distributed.
That is to say, each single ui follows a normal distribution with an expected value
of zero and a variance of sigma square u, so we have the same and constant variance
for all the ui's.
Obviously, these two properties here reflect assumptions we've made already earlier.
We've made already the assumption that the errors have a mean of zero, this was assumption
A1, which is thereby implicitly contained in assumption A5 by saying that the normal
distribution of the ui's has an expected value of zero, and we have assumed that the errors
are homoscedastic.
That is to say that they all have the same variance sigma square u, so the variance does
not vary with index i, but rather all ui's have the same variance.
This was assumption A3, which we had there, and we also had assumption A4, which said
that the ui's are all uncorrelated with each other, here it is even a little stronger by
saying that they should all be independent of each other.
In some sense, A5 summarizes assumptions A1, A3, and A4, and adds a new property, which
is that the distribution of the errors is actually normal.
I write the normality assumption here for a single observation ui, so this is a scalar,
this scalar ui follows a scalar normal distribution, and I say that this shall be the case for
all i.
Here, I write it alternatively as an assumption on the whole vector of the u's, so that's
a n by 1 vector u here, and this n by 1 vector u follows a vector normal distribution with
an expected value of 0, so that's also an n by 1 vector of 0's here, and a covariance
matrix sigma square u, where sigma square u is a scalar, like which is here, so the
same symbol, times the identity matrix of type n, so an n-dimensional identity matrix,
which again summarizes the assumptions A3 and A4.
Now we've talked about the normal distribution already, and I told you when I covered this
in the lecture that one important property of the normal distribution is the fact that
linear combinations of normally distributed variables are also normally distributed, so
we will make use of this assumption now by looking at the least squares estimator.
As you recall, the estimator is a random variable, that's it from estimate, which would just
be number, so this is a random variable here, beta hat, which of course also has a distribution.
We can now conclude from the property that the errors are normally distributed, that
the least squares estimator is also normally distributed, because we know beta hat is x
prime x inverse e x prime times y, this term here in parentheses is y, this is equal to
x beta plus u, so in the way we've done this already many times now, multiplying out this
expression here, we know beta hat is equal to beta, because x prime x inverse e times
x prime x is the identity matrix, so this gives us just a beta here, the true value
of the parameter, the unknown true value beta, which we want to estimate, plus x prime x
inverse e times x prime u, so this is sort of the random component of the estimator beta
hat, and you see now that this is a term which is linear in u, think of the axis as being
exogenous, possibly non-random matrices, then we can take the x prime x inverse e x prime
expression here, just as some type of constant by which the u is being multiplied. The u
appears in this expression just in linear form, so there is nowhere u squared or ui squared in
this expression, so this is just linear expression which we have here, and since we now know by
assumption a5 that u is normally distributed, it follows that beta hat is also normally distributed
with expected value of beta, so beta hat as we know is unbiased, and with an error term here x
prime x inverse e x prime u, whose covariance matrix is sigma square u times x prime x inverse e,
as we have computed this in the last lecture, so if you don't recall, please try to reproduce
the result. The covariance matrix of this term here is sigma square u times x prime x inverse e,
so summarizing what I've just said, we know that the least squares estimator beta hat follows a
normal distribution with expected value of beta, so of the true parameter, and covariance matrix
sigma square u times x prime x inverse e. Now let us denote the typical element of x prime x
inverse e by cij, so obviously this is some complicated expression here, which is rather
difficult to write down in terms of the x observations, as of the observations we have
in our regressor matrix, because we have to take the square here and then take the inverse of that,
so we just denote the typical element of x prime x inverse e by cij, where i is the row index and j
is the column index. Then this notation here implies that each single coefficient in the
estimator, or each single component I should say of the estimator, let's say the kth component of
this vector beta hat here, would be beta hat k, and that this is also normally distributed
with expectation of beta k, the true value of the parameter vector, and then a variance
of sigma square u times ckk, so that is the kth element on the main diagonal of the x prime x
inverse e matrix. So this here would be the variance of beta hat k, this is the true
variance of beta hat k, because here we still use sigma square u as if we knew the true value of
sigma square u. Obviously we usually do not know what the true value of sigma square u is, and then
we have to estimate what sigma square u is, we talked about the estimator in the last lecture,
but the true distribution of course relates to the true value of sigma square u. So while we
write a hat here to denote the estimator, this here is just a real number, it's the true but
unfortunately unknown value of sigma square u, as this is also the true but unfortunately unknown
value of beta k, these characterize the distribution and when we want to do hypothesis
testing we will have to replace this value here by its estimate and this value here also by its
estimate depending on the question we want to analyze in our hypothesis testing.
Now from the fact that we now know what the distribution of the estimator is, we know that
the coefficient estimator beta hat k is included in a interval which may be defined as the true
coefficient unknown for us, true coefficient minus 1.96 sigma u times the square root of ckk,
so that's the standard deviation of course of beta hat k and the same thing with plus 1.96
times the standard deviation of beta k hat. Okay so this is like almost already like confidence
interval, it is not yet the confidence interval but we'll derive it from this expression here
knowing that the estimator is contained in this interval here with the probability of 95 percent.
So equivalently we may write that beta hat k minus beta k minus the true value is contained
in an interval which is plus minus 1.96 times the true standard deviation of the beta hat k
or writing this now in standardized magnitudes we can define a magnitude zk as beta hat k minus
beta k divided by the standard deviation sigma u times square root of ckk which would be contained
in the interval minus 1.96 and plus 1.96 since we know that this magnitude here zk follows a
standard normal distribution so zk is distributed as n of zero one right the expected value of
this term here is zero because the expected value of the numerator is already zero and we have
standardized by dividing through by the true standard deviation so variance and standard
deviation are one for the standardized variable zk. Problem number one is that we do not know what
the true value of parameter beta k is so we cannot actually compute that k since here is
beta k contained in zk so since we don't know this value here zk is actually unknown to us
however what we can do is that we can formulate hypotheses about the true value of beta k so we
can say let us assume that beta k is some beta k zero so that would be the null hypothesis h naught
beta k is equal to beta k zero and assuming that this null hypothesis is true we can then compute
zk in this way using beta k zero up here in the numerator of zk.
So when we want to test this hypothesis here we would say our test rejects the null hypothesis
if the estimate beta hat k is not too far off the hypothesized value beta k zero
which means that at a 5% significance level we should see that zk zero is between minus
minus 1.96 and plus 1.96 if zk zero is defined as beta hat k minus beta k zero minus the hypothesized
value of the true parameter so minus the value which we assume to be
and test to be the true value of the parameter vector.
Problem number two as I already said is that we do not observe zk or we're still unable to compute
either zk zero or zk in this form here because we do not know what c u is we do know the square root
of the ck case because this depends on the observations x only and we have the observations
x but we do not know what sigma u is sigma square u. So second problem is that in general we ignore
sigma u and then of course we will replace sigma u by its estimate and the estimate of sigma u as
I have laid out last lecture is or the best estimate is actually u hat prime u hat divided
by n minus k and now of course we take the square root here because we do not want to estimate the
variance and we want to estimate in this case standard deviation. So this here would be the
appropriate estimate to use because it's the unbiased estimate on this thing here the unbiased
estimate of the variance. Now as we have also already discussed in the section on hypothesis
testing in slides the two of this lecture the standardized ratio beta hat k minus beta k divided
by sigma hat note the hat here sigma hat u times square root of ckk is not normally distributed
anymore because now we have left the world of linear transformations of u beta hat k was still
just a linear transformation of u so we know that beta hat k the estimator of beta k is normally
distributed but now we divide through by a parameter or by an estimator which is not a linear
transformation of u anymore you see there is the sum of squared u's involved here and then we take
the square root of the sum of squared u's so this does not cancel the squaring since not each single
observation is taken back to its square root but it's sum of squared values of which we take the
square root here so that's a highly non-linear transformation of the u's which we do here
and dividing one linear transformation of the u's by some non-linear transformation of the u's gives
us something which is not a linear transformation anymore so this standardized value is not normally
distributed anymore tk is not normally distributed while zk was indeed normally distributed
while tk is actually just an estimate of the unknown zk the distributional assumptions
change and we move of course to a t distribution here as we have also already discussed in a
previous lecture so tk follows the standard t distribution with n minus k degrees of freedom
so it's possible to show this i won't show it to you i won't prove it to you you just have to
leave it but it is as we have already discussed this in the previous lecture on hypothesis
now when we hypothesize a particular value of beta k the standard hypothesis the test
is of course that the true parameter is zero so that a regressor is not important regression
and the true value of the parameter is zero then this means this particular kth regressor
we should rather not include in our regression because it has no explanatory content so the
standard test which we do in regression output is to test the null hypothesis that beta k is equal
to zero so this would be this null hypothesis here h zero is that beta k is equal to zero
and therefore the test statistic now written as the t statistic which we can observe which we
can compute tk would be just the coefficient estimate divided by its standard error or better
by its estimated standard error because we use the estimate of the standard deviation of the use here
now this t statistic has of course a certain p value which is just the probability for the null
hypothesis to be true and we know where the p value comes from namely from the student t
distribution most regress regression packages that i know of give the regression output in such a
form that they report both the t statistic and the p value for each estimated coefficient
which is in some way of course redundant because we know the t statistic then the p value follows
immediately so if we know that the t statistic is greater than 1.96 we know that the statistic
is significant at the five percent level and if it is smaller than that then it is insignificant
at the five percent level so in principle the p values are not really necessary but often people
are interested in the p value on its own right in order to see whether it is well perhaps just eight
percent and then you're saying it's not really significant but the likelihood that the null
hypothesis is true is rather small just eight percent so perhaps i do let the i do leave the
regressor in the regression sometimes people do do say something like that so they are sometimes
interested in the p value for judgmental issues however as a matter of experience usually does
not make a great difference if you have a regressor regressor in a regression with say a p value of
eight percent so somewhat above five percent or if you don't have it in the regression usually
most other coefficient estimates do not change by much and the significance levels do not change
by much and the r square doesn't change by much if you just pick out such a weak regressor from
regression even at the five percent level the explanatory content is usually rather low so
changing the specification and eliminating a regressor which is just barely significant at
the five percent level does also in most cases not make a big difference with respect to the
explanatory power of the regression as you know the rule of thumb is that a regressor is significant
if its p statistic is greater than two if you don't want to bother about 1.96 then let's use
two as a rule of thumb now this explains this block of regression output here you see the t
statistics here and you now know how they have been computed namely that's the coefficient estimate
so this value here that's the beta hat k or for the second regressor this is the beta hat k
divided by its standard error and the standard error is of course the square root of
the variance of the coefficient estimate so it's the sigma hat u times the square root of the ckk
if you divide this coefficient by the standard error here you arrive at the t statistic 4.28
and if you divide this coefficient by standard error then you arrive at the t statistic of 43
and you can use the t distribution which is tabulated in most statistical textbooks
to find out that for these two t statistics here the probability that the true coefficient is
actually zero this probability is zero so we can reject the null hypothesis in favor
of a hypothesis which says these coefficients here are different from zero so now you are in
a position to understand exactly how this output here is generated and how it is related to each
other you know what the r squared and the adjusted r squared is you know what the standard error of
the regression is and where it comes from how it is computed of course you know what the sum of
squared residuals is namely the objective function which try to minimize and least squares estimation
log likelihood i have not yet explained and i will not explain for some time
how this is exactly computed well basically we set up the likelihood function and then
insert the sum of squared errors we will still talk about the f statistic here and about the
distribution the probability of the f statistic so that's actually the next step in explaining
regression output here the same thing for our regression on stock prices which i have
introduced last time so dependent variable is the index of stock prices standard and course
index of stock prices and we have 210 observations here we have three regressors in this regression
constant linear trend and total factor productivity we see the t statistics which are computed in
exactly the same way as i just explained for the previous regression and we see the associated
probabilities level probability levels which here are not all simply zero in particular
the last regressor on total factor productivity which is the only economic regressor constant
and t being more or less some kind of mechanical regressors included in this regression here
which is not exactly significant as you see it is very slightly above five percent so the probability
of the null hypothesis being true is five point four percent null hypothesis being true true
meaning that this coefficient here is possibly just zero and total factor productivity doesn't
add anything to the explanation of stock prices you also see it from the t statistic here which
is just a little lower in absolute value than one point nine six that's one point nine four
in this case so it is very marginally not significant at the five percent
i alerted you to the fact that this regression is not very well specified and actually the
residuals in this regression here the u's are certainly not independent further tests would
reveal this those of you who know what the term what's the statistic is we'll cover this a little
later i see this immediately from this value of the term what's the statistic which should actually
be somewhere between one point six and two point four and here it is far off one point six zero
point zero two so there's very high and strong autocorrelation in the residual as the germ
watson statistic informs us so all the assumptions on normality are certainly violated in this
regression here and that's another reason why we cannot trust these results here but taking
at face value and assuming that assumption a5 does hold an assumption which is immediately
falsified by the term what's the statistic as we later understand assuming nevertheless
that assumption a5 is satisfied we would here say that tfp is just barely not significant
now recall that in hypothesis testing there are typically two possible errors which we can
encounter one is the type one error which is i repeat this here to reject the null hypothesis
although the null hypothesis is true and this probability of a type one error is always equal
to the significance level alpha because we reject null hypothesis when we have a p value lower than
alpha for the null hypothesis so alpha exactly indicates the probability for rejection of the
null in the case in which the null hypothesis is true the type two error is sort of the complement
of type one errors that would be the failure to reject the null hypothesis although the null
hypothesis is wrong which of course can also be the case now the type two errors are more
more complicated than the type one errors because to compute the probability of a type two error
would actually require knowledge of the true value of the parameter which we test so let's say under
h zero under the null hypothesis we had assumed that the true value of the parameter is zero
now suppose this null hypothesis is wrong true value of the parameter is not zero
but the probability that we fail to reject the null hypothesis then of course depends on
how close the true parameter is to zero so for instance when the true parameter is very close
to zero but different from zero nevertheless then the probability of failure to reject
h zero is of course very different from the case where the true parameter would be far off
zero so where the null hypothesis is very clearly not true as far off reality so that would be a
different case from the case where perhaps the true parameter is not exactly zero but very very close
to zero in general it is true that the probability of a type two error is the greater the smaller
the probability of a type one error is sorry that's a misprint here that should be type one error
here or the other way around right the probability of a type one error is the greater the smaller
the probability of a type two error is so the two of them are exactly complements you can either
write the type one here and the type two here or the type one here and the type two here but
but you cannot as i have done here erroneously excuse the mistake write type two errors in both
cases that does of course not make sense and i would correct it okay moreover the probability
of a type two error depends on the true parameter as i have already explained up here so as i said
if the true parameter is far off the hypothesized parameter then the probability of a two of a type
two error is rather small there are two terms which you may encounter in reading econometrics
literature which you should know of sometimes people speak of the level of it the level of
a test is the probability of a type one error and sort of the way you can memorize that is that we
speak of the significance level right we know the significance level is the probability of a type one
error so that's the level of a test and such a level of a test should ideally be rather small
say we choose five percent or one percent or something like this so low percentage
we cannot however just take a very very small probability level because
the smaller this level is the smaller is also the power of a test and this is the second term
i want to acquaint you with if you are not yet acquainted to this term here the power of a test
is the probability to reject H0 if the null hypothesis is wrong so this is the probability
of a type two error right the level of a test is the probability of a type one error and the
power of a test is the probability of a type two error obviously we want from any test that the
power of this test is large so we want that the probability of a rejection of the null hypothesis
if the null hypothesis is actually wrong that this probability is high
now the power of a test and the level of a test must be balanced and there's always a trade-off
between the two so the lower the level of the test is which is a general good to have a low
level of the test the less unfortunately will be the power of the test and the higher we drive the
power of a test well the higher is the probability of a type one error so the higher is the level
and we don't want a higher high level so some compromise must be drawn between the level of
a test and the power of a test and when people devise new tests in the econometrics literature
then they usually will report both the level and the power of a test and they will actually
argue for certain specifications of the test in order to achieve a satisfactory power for a given
level of the test or they will fix the level of the test say at five percent and say now let's see
how we can devise the test in such a way that the power is maximal is as best as possible
so that's some kind of optimality criterion when designing new tests
we will not design any new tests here we will just use standard tests one of which is the T-test
which you know of which we have already talked about now as the most basic test to be applied
to regression results so let's continue the discussion of T-statistics here
by repeating what I have already said on confidence intervals but now to familiarize
you with the regression framework let's do it in terms of regression theory so as you know for
a test level alpha for a significance level alpha we would have that the test statistic Tk
which we can compute should lie between two critical values which I denote here as T index
n minus k alpha however negative of that positive of that value so the true test statistic which
we compute from the regression results should lie between the two critical values here with
probability one minus alpha because there would be just probability of alpha over two for a test
statistic which is even smaller than the negative of this critical value or the same probability
alpha over two for Tk being greater than this critical value here so the probability for Tk
to lie between these two critical values here is just one minus alpha and obviously the critical
values are then taken from a t distribution with n minus k degrees of freedom and obviously we have
a two-sided test here so that we have the residual probability alpha over two for very small
and same probability alpha over two for very large values of the test now Tk is of course
just this expression here which we have already derived the estimator beta hat k for the kth
coefficient of parameter vector beta minus beta k minus the true value which we do not
know but on which we can have certain ideas where we have certain hypotheses what the true value
of beta k may be divided by the true standard error here sorry this is not this is not the t
here because i do not use the true because i use the true standard deviation of the u c s sigma u
and not the headed value so that's not exactly correct what i have written here which i think
i should change the formulas and insert a hat here that makes more sense so that's another typo in
the slides for which i apologize so please correct this for yourself and i will upload a corrected
file here that should be sigma hat u here and here and there right and then in the usual way
when we write it this way here we can transform these two inequalities such that we know that
basically the value of the estimator beta hat k plus the standard deviation times the critical
value must be greater or equal than the true parameter value and this is greater or equal
than beta k minus the test the the critical value times one standard error here so this
expression 36 would describe as a confidence interval for beta k i would tell us that in 95
of the cases the true parameter beta k would lie in between these two bounds which i have stated
here with sigma hat u rather than sigma u in the formula
you can verify what i have just said this thing by doing another MATLAB exercise much in the spirit
of previous MATLAB exercises that i gave to you so again generate some matrix of random numbers
let's say 100 observations and three regressors in the regressor matrix x and then again you
run a thousand regressions of a particular type so you loop over a thousand regressions
in where in each of the in each of the loops you generate a random vector of disturbances u j
which follow a normal distribution so that so that would be important in this case
that you actually use a random number generator which generates normal normally distributed
distributions and then you compute in each of the thousand loops your dependent variable y j
from the following regression so x times parameters 1 1 and 0.1 plus u j so we would
know that the true parameter vector beta is equal to 1 1 and 0.1 and obviously you may now
test in each of the regress regressions whether this parameter parameter is actually zero
so that would be testing a null hypothesis which is not true right but from the regression results
it may be that you think perhaps the last coefficient is actually zero and not 0.1 so you
may actually run now tests in each loop of this sample program here which tests the hypothesis
that the third coefficient beta three is equal to zero you know by construction that the hypothesis
is not true so you test a null hypothesis which is not true and you should now check whether your
results reject the wrong null hypothesis in approximately five percent of the cases
right no not in approximately five percent in the cases you should
check how often with what probability the beta three coefficient here is detected as being
non-zero if you test at the five percent level of significance so you would study the power of
the test in this exercise here you would find out what the probability is to reject the null
hypothesis of which you know that it is not true and thereby you can get a feeling what the power
of the test is if the if we know that the true parameter is 0.1 you may also when you write such
a program check how the power changes when you use other values for beta beta three so not use 0.1
but use 0.2 or 0.05 or whatever you can then see how the power of the test changes with the true
parameter which you use in this regression here. I think that was why it was just confused that by
some mistake I skipped this exercise here when I changed the slides so actually we had discussed
slide page 95 here and I should first have come to this exercise which relates to the level of a test
and then comes the next exercise basically modification of the first one which I have
just introduced which relates to the power to the power of a test so let me go back one slide here
the first exercise actually that I recommend is of the same design but using a value of 0 for beta
three so in this case we would know that the third regressor which we have in our regressor matrix
x is actually not explaining or is not related to the dependent variable yj because the true
value of the parameter is zero but if you test this third regressor never nevertheless so you
estimate the regression with three regressors not knowing that the third regressor is actually not
needed then you can run the t-test at the five percent level and you can check how often in your
thousand replications the hypothesis beta three is rejected the hypothesis beta three equal to zero
is rejected by the data and it should actually be the case that only in approximately five percent
of your thousand observations so only for about 50 of this thousand loops you would reject the
null hypothesis that beta is equal to that beta three is equal to zero and in 950 cases you should
accept this hypothesis so you would get a feeling of what a type one error is if you do this
exercise and then here the same exercise is repeated with a non-zero value for beta three
zero point one as I have just explained and that would give you a feeling what the power
of a test is and how it changes when you change this coefficient here
now what we have talked about is testing one particular coefficient beta k with a hypothesis
of beta k being equal to zero suppose now that we have a joint hypothesis so suppose that we
want to test an hypothesis for a regression equation which says that more than one coefficient
is equal to zero so it could be a hypothesis of the following form another hypothesis would be
that beta k minus j plus one up to beta capital k are all equal to zero or it's perhaps easier to
see what I mean by looking at equation 38 we have the typical regression model y is equal to
x beta plus u and now we split the x matrix column by y's so the first k minus j columns
are in matrix x zero and the next j columns are in matrix x one so we just split here vertically
basically um the this vertical or is this horizontal I don't know so we split along the
columns so that the first k minus j columns of matrix x are matrix x zero and the last k the
last j columns of x one are in matrix x one and we split the beta vector accordingly so that matrix
x zero is multiplied by parameter subvector beta zero and matrix x one is multiplied by
subvector beta one so the vector beta zero would have k minus j by would be of type k minus j by
one so one column k minus j rows and the subvector beta would be j rows and one column of course
plus error term now the hypothesis that we would like to test is that this vector beta one here
is actually identical to zero so that all the regressors contained in matrix x one are actually
not needed and have no explanatory power right so that's the null hypothesis of a joint hypothesis
on coefficients in this case on j coefficients in our regressor matrix how would we estimate
or sorry how would we test such an hypothesis well the first thing would be that we estimate
the usual regression so just compute beta hat in the usual way as the least squares estimator beta
x prime x inverse times x prime y would be beta hat we call this the unrestricted regression
because there are no restriction placed on our regressor matrix here we just use all the regressor
all the regressors even though we know that some of them are actually superfluous and we need them
but we have them included in our regressor matrix nevertheless and then just compute
usual least squares regression denote the estimator by beta hat so no restrictions on beta
one here and not imposing the restriction that beta one is actually equal to zero as
h zero specifies that to be the case now for the least squares estimator denote by s one the sum
of squared residuals u hat prime u hat is the sum of squared residuals as you know so s one is
u hat prime u hat or you can also write it as y minus x beta hat prime y minus x beta that's the
sum of squared residuals for the unrestricted regression and now we re-estimate the regression
under the restriction of the null hypothesis so under the restriction that beta one is equal to
zero what does this mean if beta one is equal to zero then we can impose the restriction on the
regression by not including matrix x one in our regression right if beta one is equal to zero then
x one times beta one is equal to zero so the regression model in this case with this restriction
being imposed is just that y is equal to x zero times beta zero plus u and this term here would be
zero um sorry so we would have this model here this is the restricted model which imposes the
restriction beta one is equal to zero and this gives us an estimate which i here denote by beta
zero tilde right which is actually the least squares estimate of the least squares estimator
of the restricted model so it's x zero prime x zero inverse times x zero prime y and the
corresponding sum of squared residuals would be s zero and this would be uh utility prime
utility so utility are the residuals from this regression here and we can write this as y minus
x zero beta zero tilde prime the same expression not transposed so s one is the sum of squared errors
from the unrestricted regression s zero is the sum of squared errors from the restricted
regression and you can memorize the significance of the zero index here and the one index here
by saying well this corresponds to the null hypothesis this is why i have an index of null
here of zero here right so this sum of squared residuals corresponds to the null hypothesis
and this here corresponds to h one to the alternative hypothesis that at least one of the
beta one coefficients is different from zero
now the question that i would usually ask you now if we had normal lectures would be
what do we know about s one and s zero in particular can we state that one of these
magnitudes here is greater or smaller than the other in all possible specifications of the
regressor matrix x does anyone know the correct answer for this question then please raise your
hand do you have any idea whether s one or s zero is systematically greater or smaller
than the other sum of squared residuals
please wave to me raise your hand if you want to answer the question
i don't see any sign so the answer is easy sorry oh yeah there's somebody
uh please um yeah uh there comes the answer in the chat
a zero is smaller um why do you think that a zero is smaller
because the results are as small as possible no that's unfortunately not
the correct answer wait a minute i have to go back to the right slide
so the true answer is actually that s zero is always larger than s one so just the opposite
of what you just tried to say and well yeah my question is why is this the case now given that
i told you what the true the correct answer is uh what do you think again about it why is it true
that s zero sum of squared residuals in the restricted regression is always greater than
or equal to s one
we'll observe that in both regressions regression one and regression zero if i may call them so
we want to explain the same dependent variable right we want to explain why
and in uh equation 39 here which gives rise to sum of squared residuals s zero we are
able to explain why by choosing coefficient vectors beta zero freely basically we may choose
any value of beta zero such that it minimizes you whereas in equation one we may we may minimize
the sum of squared residuals choosing any value for the beta zero so we may just make the same
choice as before as in the restricted regression plus we are able to choose beta one freely in the
restricted regression we are not able to choose beta one freely because we have restricted the
beta one coefficients to be all zero obviously in the unrestricted regression we are free to choose
the beta ones as being equal to zero if this should be optimal if this should minimize the
sum of squared residuals so it's completely possible to reproduce exactly the same estimate
that we would have in the restricted regression but we may also choose non-zero coefficients
for the beta ones so we have more flexibility in this regression because we can choose any
value of beta one which allows us to minimize the squared errors of this regression
so when we have more flexibility we can of course achieve a better value a smaller value for the
sum of squared residuals so the minimum will be smaller in the unrestricted regression that will
be in the restricted regression and because of this it is always true that s zero is greater than
or equal to s one all right so we can explain less in the restricted regression than we can in
the unrestricted regression so the difference s one minus s one is always non-negative right
it's positive or at best case even though this never happens it would be exactly equal to zero
but it's certainly not negative cannot be the case now we know let us for simplicity say this term
here is positive we know this term is positive but the question actually is is it significantly
different from zero so is this positive value here uh large enough to differ significantly
from a value of zero because if it were not significantly different from zero then we would
say okay it is may it may still be true that all the beta coefficients are uh all the coefficients
for the beta one vector are actually equal to zero uh since we don't really make much progress
in reducing the sum of squared residuals when we move to the flexible approach in which the beta
one coefficients can be chosen in a way which is uh which which would give non-zero values for the
beta one coefficients so we want to take and actually want to formally test whether the
difference between s zero and s one is significantly different from zero and the way to do this is to
construct a statistic which i hear is called small f which actually will give rise to an f test which
i usually write with a capital f here but this is just the test statistic that's not yet the
distribution that's why i distinguish it in notation here's a small f here now how is such
an f statistic constructed well the idea is not as complicated as it may perhaps a first
here to you when you are introduced to f tests with two different numbers on the degrees of
freedoms what we look at is a ratio so this here indicates this ratio we have something which is
in the numerator of the ratio which is s zero minus s one over j and we have something which
is in the denominator of this ratio which is s one over n minus k what do we do here why we
choose these funny looking uh numerators and denominators where we try to standardize our
results and perhaps it is best to start with the denominator here s one over n minus k
because what does this exactly tell us s one is the sum of squared residuals uh as you know
in the unrestricted regression and here i uh divide this by the number of degrees of freedom
so i i divide by the degrees of freedom by n minus k so basically um you can think of this as
the average squared error that we have for each degree of freedom in our regression if it is
unrestricted and now we would like to know how much additional squared error do we get when we
introduce restrictions and here it is of course clear that the more restrictions sorry the more
restrictions we place on our model so the more parameters are restricted to be equal to zero
the higher will be the value of squared residual residuals which we obtain in the restricted
regression so it makes sense to divide the difference between s zero and s one by the number
of restrictions so in a sense this here tells us the increase in the value of the squared residuals
per restriction the average increase in the value of the squared residuals
when we add one additional restriction or when we set one of the coefficients equal to zero
exogenously right so that's the average error which we commit or the average squared error
which we commit per restricted coefficient and this we relate to the average error which we
have per observation and then we see whether this error here the average squared error per
restriction is significantly greater than the average error per observation which we have had
in the unrestricted regression so that's the principal idea of an f statistic to see how much
additional error variants do we get when we impose additional restrictions on the regression
and we compare the additional error variants with yeah well what is this here this is basically
the estimator of the variance right this is the unbiased variance estimator sigma square
u hat s one over n minus k the numerator in this expression here is just the unbiased
variance estimator sigma square u hat so basically we ask here how much additional
variance do we get per imposed restrictions restriction relative to the level of variance
which we already have in the unrestricted regression this is why this is a sensible
test statistic which we have here and the distribution of this test statistic follows an
f distribution which can be shown but which i will not show in this lecture
it's not that trivial to show it but in general it is true that if we look at the ratio of two
numbers where each number is the sum of squared random variables and each of the squared random
variables is normally distributed then the ratio of such random variables is distributed
as an f distribution with particular degrees of freedom provided that the numerator is independent
of the denominator which i would have to show and which i will not show that this is the case
but you can show it then you have the result that distribution of this test statistic here
follows an f distribution so perhaps just memorize if you have the ratio of the sums of
squared normally distributed variables this divided by each other and granted that
numerator and denominator are independent of each other then the variable is f
so what i've just said i just summarized one sentence on this slide here it is possible to
show that f follows a standard capital f distribution with j and n minus k degrees of
freedom so j degrees of freedom in the numerator and n minus k degrees of freedom in the denominator
of the test statistic f and then obviously and like for any test we reject the null hypothesis
when f is large sorry this was my fault we reject the null hypothesis when f is too large
so in this case we would accept the alternative hypothesis h1
what does the alternative hypothesis h1 state well it basically tells us that there is at least
one coefficient beta j in the coefficient vector beta 1 which is different from zero
so when our null hypothesis was actually that this beta 1 vector here was equal to zero
it means that all of the components in beta 1 are equal to zero so the null hypothesis tells
us that one of these components at least one of these components is non-zero right maybe that all
the others are still zero but at least one of them is non-zero this would be the alternative
hypothesis and this alternative hypothesis h1 i have stated here says that one of the beta j's
for j running from k minus j plus 1 up to k that one of these beta j's is non-zero
now in the standard regression output which you get from econometrics packages there's always
something reported on the f statistic and never do they state in the regression output which
precise hypothesis is being tested because obviously there are many ways to split a
regressor matrix like here between x0 and x1 many ways to have a hypothesis in such a way
that a particular subset of the coefficients is equal to zero so the standard f test is actually
the f test which tests the hypothesis that all the coefficients in the beta vector are zero
with the exception of the first one which is the coefficient on the constant
because in almost any regression we know that researchers will include constant there so that's
a regressor which almost reliably is in the progression or the other regresses may vary
depending on the research topic so the standard f statistic reported in regression output always
tests the hypothesis that with the exception of the constant as the first regressor all other
beta coefficients are equal to zero this is what i have written here so the standard non-hypothesis
in regression output is that beta two up to beta capital k are all equal to zero and just beta one
is allowed to be non-zero and beta one is usually the constant
so this is what you see here in our consumption income regression the f statistic is enormous
one thousand eight hundred and thirty and the probability of the f statistic is zero
so the probability that log y is actually equal to zero or the coefficient of log y is actually equal
to zero is zero as we knew already from the t state test because in this particular case the
f statistic doesn't really test a different hypothesis than t test for this coefficient
tests since we just have two regressors here and the f statistic tests the hypothesis
that this thing here is equal that the coefficient on log y is equal to zero
could again an exercise in this context reproduce the f statistic for the benchmark regression
uh benchmark regression meaning log private consumption request of log income reproduce
the f statistic and its p value manually so by computing it from hand in order to see
that you have understood what the f statistic actually means
any questions which relate to hypothesis testing
this is not the case so we move on to asymptotic properties again the numbering should be 3.5
and now what we want to do is that we want to look at the least squares estimator when the
assumptions a1 to a5 or some of these assumptions are actually not satisfied i told you already
some of the assumptions in particular assumption a2 a3 and a4 are rather strong assumptions
oh well a5 is actually also strong so it is legitimate to raise the question of what happens
if a1 a2 a3 a4 a5 some of them are actually not satisfied and of some help when we are
faced with scenarios in which we have to believe that these assumptions a1 to a5 are not satisfied
it is of help to resort to asymptotic properties of the estimator because we will usually not be
able to derive finite sample properties anymore when a1 to a5 are not satisfied or at least a1 to
a4 are not satisfied so let's look at what happens asymptotically first have a look at the first two
lines in this expression here these are transformations of the least squares estimator which
you already know beta hat is equal to two parameter vector plus x prime x inverse times x prime u
so this we are familiar with this property now what we have done so far is that we have usually
argued where x prime u has an expected value of zero right and the assumption which we needed
for this was the strict exogeneity of x so we needed to have the assumption that the expectation
of x prime u is given x is actually equal to the expectation of u which would be zero
and then the whole thing here is zero but this conditional independence which we had assumed
here the effect that the expectation of x prime u is equal to zero this expectation
this assumption is a strong assumption because it implies that the u here is no way related to
the x matrix here and this is not necessarily true in particular in time series context
it is not true for instance it is often the case that in a time series context
the x matrix contains lagged values of the dependent variable so suppose we want to explain
y then we would have lagged values of y in the x matrix well if we have lagged values of y in the
x matrix then we know of course that lagged values of u of the errors are also in the x matrix
because y is equal to x beta plus u so when we have lagged values of y in the regressor matrix
then we also have lagged values of u in the regressor matrix so we would have lagged values
of u in here and we would have the non-lagged values of u here but since we multiply out this
matrix product here we would get terms in which u a value of u is multiplied by its own lagged
values and it is not clear that this is always that the expectation of this is always zero
or we would have perhaps this is easier to see then we would have lagged values of y multiplied
with lagged values of u or in a different row of this u vector here which would have a expectation
which is non-zero because this involves the variance of the u's actually that's the better
way to frame it i think the first burning was was not so exact so that would be the the right way
to look at it if you have a lagged y here let's say in row two then this depends on a lagged error
and this lagged error would be in this vector of all the u values which we occur with which we
observe or which we implicitly observe in our sample so that there would exist a correlation
of this lagged u term here with the corresponding different row in this u matrix in this u vector
here so that we would when multiplying out x prime u encounter a term which is squared u i
and this would have an expectation of sigma square u so it would be non-zero therefore this
term you would not vanish basically what we want to do now is that we want to boil down
our assumption on the x prime u on the strict exogeneity of the axis here to an assumption
which is weaker in the sense that we only want to have the property that the axis in the particular
particular row of x are independent of the u disturbance in the same row
but not independent of all the u disturbances in this vector so on all rows in the u vector
and in order to do this it is helpful to write the x prime x inverse representation here
and the x prime u expectation here in terms of vectors which we add up so
at the beginning of the regression model i had already informed you or asked you in an exercise
to verify that x prime x is actually the same thing as the sum over all observations of x i
times x i prime where x i are just the explanatory values for the ith observation
right these are just the explanatory variables for the ith observation here
and i multiply the this vector here by its own cross product so this x i is a k by one vector
being multiplied by one by k vector which gives me then that the product here is k by k
exactly as x prime x is k by k and well then this is added up over n observations so n k by k
matrices added up is still in a k by k matrix so x prime x is the same thing as this expression here
and then i have to take the inverse of this expression and the same thing holds true for
x prime u x prime u can be written as x i times the scalar value u i x i is a k by one vector
and this is being multiplied by the observation u i so we get the product x prime u which as you
know is a k by one vector since x has k columns x prime has k rows so this is k times n or k by n
matrix multiplied by n by one matrix gives us a k by one matrix so we here also have a k by one
matrix since x i is a vector with k rows being multiplied by a scalar is still a vector of k rows
added up n times the sum of n k by one vectors is a k by one vector so it seems that the formats
here are correct the dimensions of the matrices are correct and then we can just write our matrix
products in the somewhat more complicated way of adding up vector products like i've written it
here and the importance of that is illustrated by the fact that we now see that there are just the
products of the observations of the same observations involved in these sums here so it's always x i
multiplied by u i we never have x i be multiplied by u i minus one or minus two or minus three or
something like this always the same observation index or in the time series context it would be
the same time index x t and u t or here that is also the case the correlation between different
observations only comes up when we multiply this term here with this term here and we are
are now looking for conditions under which the product of these terms here is actually zero
and therefore we will use asymptotic arguments
well what we do here is essentially that we introduce the average over these sums
so you see that this here is an inverse by the superscript minus one and this here is not so
we may actually introduce one over n here and one over n here because the inverse of the factor one
over n is n right if i pull out this factor one over n out of these parentheses here then since
this is the inverse i would have to write an n between these two matrices the inverse of one
over n is n and if i pull out the one over n out of these parentheses since this is not an
inverse it's just a one over n factor so actually pulling out these two factors here out of the
parentheses gives me n times one over n so this cancels right so clearly this type of product
here is the same as this type of product here and why do we do this well because we want to
use some type of law of large numbers we know from the law of large numbers that the sum over
independent observations converges or the the average over independent observation observations
converges to the true value asymptotically right and here now we take the average of well some
sort of squared values of the xi's over all the observations i so i running from one to n and here
we do the same thing for the product of the observations of the regressor matrix times
the error and we again compute the average of all of that so what you will do in the next lecture
is that we invoke the law of large number to argue that this thing here under certain conditions
converges to zero and we will also argue that this thing here under certain conditions converges
to a finite matrix any finite matrix so if this is any finite matrix to which it converges
and this here is zero or has as a limit as zero then the product of something finite
times something which is zero is zero and the whole terms here vanish this is what we
have as additional properties when as additional properties when the number of observations
converges to infinity or goes to infinity and we will use this next time
for today i will stop the recording