Welcome, everybody, to today's lecture on estimation and inference.
If you have any of their questions, then please raise your hand.
I remind you that at the end of this lecture in the last 15 minutes we will switch to interactive mode without recording.
So, and then we can talk to each other.
And make sure that you have the link ready, which I sent out yesterday.
I would also like to announce that the next two lectures, which are on January, three and January four, you know that today's the last lecture before Christmas, right.
Those two lectures on January three and January four won't be live because I'm traveling.
So I will record them this week and upload them to Steena, and you can access them not just in the sorry to lecture to go of course, like usual.
And you can access them anytime as soon as I have finished recording them, which will be the case, I think, probably already tomorrow.
So, you have the lectures for the first week of January then already at your disposal and then you can listen to them whenever you want to but please not later.
Then, for the January three and four sessions because you need to have sufficient time to work on your student contributions like always.
The student contributions for this week here are due only on January third. So January third Monday at nine o'clock, like usual, the student contribution for the week for which I record the lectures, lectures which won't be live, then do one week later.
So this is the usual state of affairs.
Okay, so much for this, as I say, interactive mode at the end of this lecture and the next two lectures then recorded.
I don't see any indication of questions. So let me carry on.
The last thing we did was that we talked about measurement errors as a potential distortion in estimation, where we considered the case when the measurement error occurred only in the dependent variable, which is the less problematic case as you will see now, because now we turn to a measurement
error which occurs in explanatory variable only. So do not consider the case where we have measurement errors in both the dependent and the explanatory variable, but obviously then the two effects which are isolated now just combine.
So, consider just one explanatory variable for clarity. Let's say that the true equation so the population model is given by the equation 32.
So, y is equal to beta naught times yota, so just a constant, and then beta one times some variable x star plus some error, u.
And let us suppose that because Markov assumptions A1 and A2 hold, we can also assume that the full set of cos Markov assumptions holds, and no problem with this but important in order to ensure that the estimate of this equation would be unbiased and consistent if we use OLS
that is just the existence of assumptions A1 and A2. So, if we can observe x star, then OLS estimation of this equation here would produce unbiased and consistent estimates of beta naught and beta one.
The problem is that now by assumption x star is unobserved and instead we observe x. So we observe some variable which has correlation with x star but which is not the same thing, which is actually not the true variable which we would need when looking at the population model.
So an example of that could for instance be that this here is a consumption function which we estimate on individual data, which we have, let's say from tax declarations or something like this.
So somehow we have consumption. So in this case, y would be consumption and x would be income, a very simple consumption function without any kind of forward looking behavior so more or less some type of Keynesian consumption function, old-fashioned Keynesian consumption function where x star would be income and y would be consumption.
Now the problem is that we do not have the true measure of income, but we just have reported income. So we just have income which has been reported to the tax authorities.
And very clearly it may be the case that income reported to the tax authorities is not the true income. It may very well be the case that x is actually smaller than x star because people have incentives to avoid taxes and report their income lower than it actually is.
So we have a measurement error in an explanatory variable.
Now this measurement error we described by e again. So the difference between reported income and actual income with x and x star is defined to be e.
And just to make things simple for us, I assume that the average measurement error in the population is zero. So the expectation of e is equal to zero.
This, as I have already just said, is actually not a really realistic assumption. If we deal with issues like reported income and actual income because if it is reported income, income reported to tax authorities, then it would be very reasonable to assume that the expectation of e is smaller than zero, that people try to avoid taxes.
However, it is not really a big problem if we impose the assumption that the expectation of e is equal to zero, because suppose that it were not the case that the expectation of e is some c which is different from zero, then the only thing which would actually change in our estimation is the constant term.
If this is the expectation of e, if this is non-zero, then well, okay, we can just rewrite the model, add this non-zero expectation of the measurement error term to the constant and we would estimate different constant term and have some type of modified residue which then again has expectation e of zero.
So it doesn't really play a role because the estimated value of the constant is usually not interesting. We just estimate it to adjust for means of variables, but mostly this constant is not really important.
So, while, as I say, if the measurement error comes from particular sources, we must actually expect that the expectation is non-zero, we will handle the case here just for the more simple setting where the expectation of e is assumed to be zero.
The second assumption I make is that the expectation of x prime u is equal to zero.
This is a new assumption and it's also not necessarily noxious, but we can do it in order to bias our analysis as much as possible to favorable outcomes in the sense that a favorable outcome would be unbiased estimation of coefficients or consistent estimation of coefficients.
Note that this assumption here that x prime u has an expectation of zero is an assumption which refers to the reported income, so to the variable which we observe and which we use in the place of the true variable x star.
We know already that the true variable x star has an expectation or that the expectation of x star prime u is equal to zero by assumption a2, right? We have assumed because Markov assumption a2 which actually imposed the strict exogeneity of the regressor,
in this case of x star, and then it also implies that the covariance is equal to zero, so it would also imply this property here.
But the analogous property for x does not follow from the set of Gauss-Markov assumptions, so we need to assume it separately.
And as I say, we would have to think about whether this is a reasonable assumption, but now here for the lecture, just in order to expose that a problem occurs even if we make this assumption here, we post this assumption.
So the assumption is made in order to demonstrate to you that even if we try to make our life as simple as possible, we're also assuming that reported income has a correlation of zero with the error term.
Then still we run into problems here.
So what are the properties of the OLS estimator if we now run an equation, what was it, 32, with x being used in place of x star, reported income in place of the unobservable true income x star?
Well, suppose we make the following assumption, which is sometimes called the classical errors and variables assumption, which actually is now third assumption we make on top of the Gauss-Markov assumption A1 and A2.
Suppose that for each observation i, the error ei is uncorrelated with the unobserved measure xi star.
So suppose now that the expectation of ei xi star is uncorrelated, that these two things are uncorrelated so that we can take the expectations of each factor separately
and write this as the expectation of the measurement error ei times the expectation of xi star.
And since we have assumed that the expectation of the measurement error is equal to zero, then the product here would be equal to zero, so we would know that xi star does not correlate with the measurement error.
Actually, this is again an assumption which is kind of unrealistic. And again, I make this assumption just to show you that even if we try to make our life as simple as possible
and make such unrealistic assumptions, we will still run into problems. So why is this assumption here unrealistic? We'll think about what it says. It says that the measurement error in income is uncorrelated with the true income of each in the video.
The degree by which we fail to record the income of each individual properly shall be uncorrelated with the magnitude of the income.
And that is not so very likely if we think of income reported to tax authorities because people with high income would have more incentive to under report their income because they are taxed at a higher rate.
So it may well be actually that the measurement error is correlated with the size of income. People who have low true income perhaps don't pay much taxes in the first place or don't even pay any taxes, so they would have no incentive to misreport their true income.
But people with high incomes do have these incentives, and therefore it is quite courageous assumption actually to assume this classical errors and variables assumption here.
But nevertheless, we do it, as I say, in order to show that even with this, we are far from off the hook. We still have problems with this measurement related setting key.
The one thing which you may then ask is why don't we assume also the same thing for the reported income with this perhaps solving all our problems if we have made at least two problematic assumptions already.
So why don't we add a third problematic assumption and also assume that the expectation of Ei times xi the reported income is also equal to zero.
Well, then the answer to that question is that this is impossible to assume that, right, because, look, when we have when we impose this classical errors and variables assumption, then we have the property that the covariance between true income and the measurement error is equal to zero, obviously.
But it is impossible that the covariance between the reported income, the measurement error is also equal to zero. Why? Well, the covariance between reported income and the measurement error would be, well, reported income has a mean of zero, would be the expectation of xi Ei and
well xi is the same thing as xi star plus Ei. So we would have the expectation of xi star plus Ei times Ei. So we necessarily get the term Ei squared here.
And the expectation of Ei squared is, of course, just sigma squared.
I should add, actually, I should add this to the slides here. Sorry, that I didn't state the assumption so far. I have used the assumption that xi has a mean of zero.
So basically, I look already at mean adjusted variables xi. I would already mean adjust the income which has been reported, which is obviously very easily done.
And the mean income is then captured by the constant in our regression. But only then, of course, this equality here is true.
Okay. But what is the important point here is that the covariance of xi and the measurement error is necessarily nonzero because xi is affected by exactly this measurement error here. So we cannot assume that this covariance is equal to zero.
We can assume it here, even though it is a courageous assumption, but we cannot make the same assumption here.
Which then immediately implies that we have a correlation between xi and Ei. But xi is our regressor and the Ei is part of the error term of our regression, as you will see here.
Or as you can see here, because our regression would now entail that we estimate the equation y is equal to constant beta zero times delta plus beta one x plus u minus beta one e.
Because we have substituted out the x star variable and the x star variable is equal to x minus e. So we have a new unobservatory term which is negative beta one e and that becomes part of the error term.
But now we have the correlation between the new error term, which is this term in parentheses here, and a regressor. And you know that this type of contemporaneous correlation between the regressor and the error term leads to inconsistent estimates.
And two biased estimates usually also.
Which shows you that despite of the courageous assumptions we have made, like this classical errors and variable assumption, there is correlation between regressor and error term, and we still have the problem that the estimate is inconsistent.
So since u and x are uncorrelated by assumption, right, we've made the Gauss Markov assumption that u and x, we have
not really made the Gauss Markov assumption, but we have made the assumption that u and x are uncorrelated.
The covariance between the new error term and the xi is not necessarily negative.
So when we compute this covariance here, covariance between xi, the reported income in the new error term, which includes the influence of the measurement error, then we see that we have the expectation of xi ui, we have assumed that this thing here has zero expectation.
So we are left with the expectation of negative beta one times xi ei, and this is negative beta one times sequence where e, which is different from zero.
So the OLS regression of y on x will give a biased and inconsistent estimator beta one hat, which is not satisfactory.
In fact, if we go deeper into these things, then we could formally show that the plim of beta one hat is actually the true value beta one plus the covariance between x and u minus beta one e divided by the variance of x, which is the same thing as beta one times one minus something which is positive,
which is sigma squared e, sigma squared e, the variance of the measurement error, divided by sigma squared x, the variance of reported income.
So the greater this ratio here is, the greater is our distortion. Of course, the greater is the inconsistency which we find even asymptotically.
Now, since the plim of beta one hat is less than beta one, if beta one is greater than zero, and the plim of beta one hat is greater than beta one, if beta one is less than zero, it all depends on the sign of this beta one here, we know that the plim of beta one hat is always closer to zero than the true value of beta one.
We say in such a case that the estimate is biased towards zero.
That's easier to say and better to say than to speak about the estimate is biased downward or the estimate is biased upward because downwards and upwards always you need to pay attention to the sign of the estimate.
In the case where the estimate is negative, then we know actually that the estimate is biased upwards by being closer to zero because the true value is even more negative than the estimated value, at least as far as asymptotic results are concerned.
Whereas if the estimate is positive, then the plim is biased downwards because the true value is greater than the asymptotically estimated value, beta one hat. So it's easier to speak about a bias towards zero.
In fact, we can use some identities between variances. For instance, if we define this, well, if we look at the sigma square X, which is defined just as the variance of X, which is the variance of X star plus E, then since we have assumed that X star and E are uncorrelated, this is equal to the variance of X star plus the variance of E.
That's not in general true, but just because we have assumed this, otherwise we would have a term in here plus two times the covariance between X star and E, but we have assumed that this covariance is equal to zero.
So it's equal to the variance of X star plus the variance of the measurement error. And using this identity, so plugging it into this formula here, right?
Plugging it into this formula here, we can equally well write equation 34 as the plim of beta one hat is equal to beta one times sigma X star squared, the variance of X star, divided by the variance of X star plus the variance of the measurement error,
which is the same thing as beta one multiplied by the variance of X star divided by the variance of X. The variance of X, as we know, is greater than the variance of X star, right? Because sigma squared and E is an additional term in the variance of X.
So whatever way you write it, you'll always find that the beta one is asymptotically, or that the asymptotic value of the beta one hat is equal to beta one being multiplied by something which is less than one.
Either in this form or this form, and that's just the same thing, or if you like, in this form here, it would also be less than one.
So, which tells us that the estimate is asymptotically biased towards zero, as I have already said.
And this bias towards zero in an error in variables format, so in a format where we have measurement error in the dependent variable, is called the attenuation bias in ordinary least squares.
Important lesson to learn here is that this bias occurs, even though we have made the rather optimistic assumptions and particularly the optimistic classical errors in variable assumption that the expectation of EI times X star is equal to zero.
Now, this is perhaps not as bad as it may seem from my words here, because as you see, the attenuation bias depends on the relative variances of X and X star.
So if it is the case that true income has a variance which is approximately equal to sigma X squared, then the asymptotic bias is not big.
And there would still be an asymptotic bias because as long as we have measurement error, clearly sigma squared X, sigma squared for X star is less than sigma squared for X.
But it can be that the two variances are quite close to each other, and it depends on the particular application that we have, whether we can argue that this may possibly be the case.
However, we should be aware the bias is there and we have to investigate whether we can say well the bias is perhaps not so grave, or whether we have to be afraid that it is very grave and that actually our estimate is very strongly biased even asymptotic.
So, as I say, if the variance of X star is large relative to the variance in the measurement error, then the inconsistency in OLS will be relatively small.
What I just said, variance of X star is large relative to the variance in the measurement error is the same thing as saying that the variance of X star is not much smaller than the variance of X.
So it depends on how much variation there is in X star relative to the measurement error E, which implies that measurement errors do not necessarily cause large biases.
If we think that the reported variables are close to the true variables, then we may ignore the measurement error provided that the other assumptions like the classical errors and variables, the assumptions can be justified.
So, the possibility that the measurement error has a small variance or that the variance of the true variable is rather close to the variance of the measured of the observed variable is only a partial justification to ignore measurement bias.
So we should always try to assess the magnitude of the measurement error relative to the variance of the explanatory variable.
The attenuation bias now implies that we are likely to estimate weaker effects that actually exist.
Obviously, I have to say here, just likely to estimate weaker effects than they actually exist because this attenuation bias which I have computed and presented to you on the previous slide is just an asymptotic result in finite samples, which we always have.
Clearly, the estimate need not display the same type of attenuation bias as the one that occurs asymptotically, so it may actually be that the effect which we estimate is infinite samples, not weaker than the true effect, but by chance, even stronger than that.
But the larger our sample is, the less likely is that we have this type of result where the attenuation bias is perhaps not leading to a weaker effect than actually exists.
Okay, any questions so far?
There are questions. Let's see.
First question is, is CE phi equals the A6.2? No, this is different. No, no, no. This is a completely different assumption. See, the classical errors and variables assumption, let me go back again.
The classical errors and various assumption, a variable assumption is an assumption about the correlation between the true regressor and the measurement.
It's not an assumption like A6.2, which was an assumption about the correlation between a regressor and the error of this regression.
This is the measurement error. This is the E here. It is not the U we make an assumption on. So the classical errors and variables assumption has nothing to do with assumption A6.2.
Why this ratio is positive, sigma E over sigma X? Well, very clearly, because both are variances, right? The variance is always positive.
So sigma E is positive divided by sigma X is positive, so it is positive. Sigma X squared, right? These are squares, so they're necessarily positive.
Let's forget about the case, the degenerate case, where variance could, in principle, be zero. For practical purposes, this is very clearly positive.
And then there is a question, can sigma E squared sigma X squared be smaller than zero? No, of course not. Same answer as previously.
Good. Any other questions?
But then let's go to the case of outliers. Outliers are very popular with some people who do econometrics because outliers is a great method to get the results which you would actually like to get.
So you have a data set and you run some regressions and you get a result which you don't like. And then some people start looking at the data and say, well, hey, this is an outlier and I remove this and this is an outlier and that doesn't fit to a theory.
So they delete certain observations and then they run the regression again and suddenly their result looks much more favorable to the theory they would like to confirm by their econometric style.
I'm polemicizing a little bit here, but there are indeed studies which suffer from this type of weakness because this is, of course, what you must never do, that you delete observations after having looked at your results and having found that your results don't please you.
And that they don't fit to your prejudices and then try to design your data set in such a way that the data set fits your presuppositions about what kind of results you would like to achieve.
So you should never do that. But people do it and often the justification for that is that certain observations are outliers.
So you should ask a little bit more critically what actually are outliers and is this a case of measurement error if we have an outlier.
So perhaps for starters, just intuitively, people often consider observations to be outliers if in a visualization of the data set, they look at the cloud of data they work with.
They look at single points which are far off the main data cloud.
And then they say, well, this is an outlier, and I, and I remove it.
But that is a very intuitive way of defining what an outlier is actually it's not an exact definition and defining more exactly what an outlier is, is much more difficult.
So, as an example, for instance, consider the following case, you consider data on personal wealth and on the intelligence of paper of the people who have assembled this personal wealth.
So for instance, you have data on the IQ.
And now you plot your data on personal wealth and on intelligence in a scatter plot.
And suppose that in your data set, there's also the observation on Mr. Trump, right.
And most people will probably agree that Mr. Trump is an outlier in a scatter plot of personal wealth on IQ or intelligence on personal wealth.
And most people probably will agree on that even Mr. Trump will agree on that, even though perhaps Mr. Trump has a different opinion on which direction the outlier takes whatever you would have somebody in this case, Mr. Trump, who is far out the usual data cloud.
And this does not say that this is not a real observation. I mean, there is Mr. Trump and he has assembled certain personal wealth with his IQ, whatever it may be.
And there is no reason to say that Mr. Trump is an outlier, at least not in the sense that we just remove Mr. Trump from the sample.
Because there are people like Mr. Trump who are very successful business wise, either it is because the intelligence is so much higher than ours, or despite of the intelligence not being so much higher than ours, whatever, it is interesting to know that this type of observation exists.
So, the bottom line of this is an outlier may be a perfectly valid observation also for Mr. Trump.
So as long as billionaires are not overrepresented in the sample which I just constructed as an example, it would be completely inappropriate to eliminate Mr. Trump or the likes of him from a data set, just because either their wealth is particularly large
or their IQ is particularly outstanding in one or the other direction.
We have to take data at face value, and we need to have really good reasons to argue that we eliminate data as outliers.
If we don't have such good reasons, then observations which are outside of the main data cloud are still valid observations and if we were to delete these outliers, then we would just distort the sample actually would have some type of selection bias,
and we would bias our results and this is precisely what some people do that they dislike the results they have achieved initially and then the bias their results by deleting certain observations which they think are outliers.
We should rather think about why such outliers exist rather than delete them.
There's another problem associated with that.
Because we cannot really distinguish between outliers and outliers.
I mean, if you look at a data set, and you have some observations which are far off the main data cloud, then, okay, you may say, these are outliers and some people remove them.
But there may also be data points in the main data cloud, which are not where they should be.
They would not detect them. Right. So if the main data cloud has quite a bit of variance along both dimensions, then it may well be that there is a data point which is, let's say, in the northeast corner of the data cloud but still within,
particularly within the data cloud, even though it would belong into the southwest for some reason. Right. And we would never be able to identify such an inlier because we don't have any theory about where the data point actually belongs to.
So any distinction between observations being outliers or not being inliers is completely arbitrary and opens the door for manipulation of results and you should not do it, except one case.
Namely, when outliers come about by measurement errors, which you can argue that they have been there, that they are there, or typing errors when entering data.
So that happens actually not so very rarely that if you have a huge data set which has been entered manually into your computer that there is a typing error.
Somebody forgot the decimal point, for instance, and suddenly a huge set of data which has, let's say, numbers between zero and one with several decimal points, you have another number which is something like 13,573 or something like that.
In such a case, you obviously know something went wrong there, and you can take resort to eliminating this outline.
If it is such a trivial error, like the one I just described, where all observations have a decimal point, several digits behind the decimal point, but just one digit before the decimal point, then mostly it is also possible to just correct the typing error and well, set the decimal point where it belongs.
So, such outliers are not valid observations unless you correct them, unless you are in a position to correct them and sometimes that's possible, then you should do it.
Sometimes it's not possible, then you should just delete the outlier if you think that the typing error or the measurement error is so great that you can't reasonably work with this observation.
Now what's the problem about outliers? The problem is that OLS as an estimator minimizes the sum of squared resilience.
So when we have an observation which is of the main data cloud, and this observation typically has a huge residual already prior to squaring the residual.
But by squaring it, it becomes even more extreme. So large residuals receive an awful lot of weight in ordinary least squares or any type of least squares estimators.
So outliers in general receive lots of weight in OLS.
And then it can be that just a single observation in, let's say, a smaller sample, a small sized sample, a single observation can greatly distort your regression results due to the fact that this observation lies way off the main data cloud.
As a matter of experience, I would say actually that this problem is often overstated.
When you have a reasonably sized sample of, let's say, between 30 and 50 observations, then you will mostly already find that even observations which lie quite a bit outside of the data cloud don't have that much of an influence.
There are actually statistical techniques which you can use to measure the influence of each single observation. This is the topic of robust statistics where influence functions are being computed, which characterize which observation has what type of influence you can then see whether this is appropriate or not.
As we do not do robust statistics here, I leave it at that remark, but if you run into papers which use these type of methods, which actually are not so very popular, then you know at least as a vague idea of what people have been doing there.
So, again, if the outlier is a valid observation like Mr. Trump and his IQ and personal wealth, maybe we can do nothing about it, then this is a big residual and it will be squared and there will be a certain influence of Mr. Trump's observation on the regression results.
But as it is a valid observation and there would be no justification for just eliminating Mr. Trump, you could just have to take the regression results then as they come out of the computer.
If, however, the outliers can be explained by measurement errors or typing errors, then you should either correct them or discard them in order to avoid these particularly large residuals then even being squared affect your estimation results greatly.
So, in general, if estimates change by a large amount, when we slightly modify our sample by, let's say, deleting one observation, we should be worried and check why this is the case, right, and if there are outliers and what type of outliers they are.
It can be very reasonable to, well, first, of course, always look at the data graphically, then run the regression with all the data included, and if your graphical inspection of the data has resulted in certain observations being far off the main data cloud,
then it can be very reasonable to also run a regression where you eliminate one or more of those data points just to see by how much the regression results change.
My guess would be mostly you'd be surprised how little regression results change, even though you think that a certain observation should distort your results greatly. Often this effect is overestimated.
OLS is actually a fairly robust method of estimation, and it is by far not as sensitive to outliers as people sometimes seem to believe.
But still, you should check, and if there is a great discrepancy between the first set of results, including all observations, and a second set of results with a slightly reduced sample, then you should think what you can do about it.
But do not mechanically trim your sample. I know that some software packages have these kind of functions, just trimming the sample and then cut off the, I don't know, lowest 5% of the observations and the highest 5% of the observations and these type of things.
I think that's rubbish. You should never do that. Don't use these kinds of routines.
If you are to discard certain observations, then it is your duty to argue for each single observation while you delete it. I present good arguments for that. Never do this mechanically, automatically by just using some software.
All right. One other thing which is noteworthy in this regard is that certain factor forms are less sensitive to outliers.
For instance, logarithmic specifications. If you have in the original data outliers, then the logarithm, as you know, shrinks observations disproportionately and big observations shrink more strongly than observations which are smaller in size.
So log specifications are recommendable also because they are robuster vis-a-vis outliers than if you use the data in a non-logarithm form.
There are also alternative estimation methods which are less sensitive to outliers, one of which is least absolute deviations, which you have possibly heard of sometimes.
Least absolute deviations is completely analogous to least squares estimation, but the estimation is not done by minimizing the squared residuals, but by minimizing the sum of the absolute values of the residuals.
That's a perfectly legitimate way to estimate things, but it is computationally much more demanding than to have the least squares estimation. Actually, you have to do this with a simplex algorithm.
So there's not just a simple formula which you can use to compute your estimates. You have to use a numerical algorithm which is a simplex algorithm in order to arrive at the least absolute deviations.
The trouble is related to the fact that at the optimum, the objective function is not differential. It has a kink there, and therefore it is kind of unstable at the optimum.
I will, in the next set of slides, come to instrumental variables regression, which may help solve the measurement error problem that perhaps just as a last remark before I turn to the next issue of bad controls.
Are there any questions here?
Apparently no questions, so let us now come to bad controls.
We have done analysis for causal questions where people, or actually we in the lecture, added covariates, so control variables to these causal regressions.
And we noted, for instance, in this Tennessee school experiment, that adding covariates reduce the residual variance and increases the precision of the estimates.
So I actually advised you to use covariates if they are good covariates in order to get precisely estimates. It's useful to have such covariates, even though the point estimate for the coefficient of interest for the causal variable should actually not change by much when we use control variables,
provided that the randomization was already successful, provided that we do not need the randomization conditional on observable variables.
Now what I would like to warn you of in this section of the lecture is that the control variables may also cause problems.
It's not always the case that more controls, more covariates, are better than fewer covariates.
And one of the reasons for that is that some variables are actually what we call bad controls, and they should not be included in a regression model, even when their inclusion might be expected to change the regression coefficients.
So what are actually bad controls, and how can we distinguish good controls from bad controls? Bad controls are, loosely speaking, just variables which are themselves affected by the treatment, by the experiment which we conduct.
So bad controls are variables which are themselves outcome variables affected by the treatment.
And this should make it already intuitively clear why we should never include such bad controls in a regression without at least thinking properly about this, because the bad controls would capture some of the treatment effect.
And we would need to know exactly is this legitimate that they capture some of the treatment effect, or is it illegitimate and distorts our regression results.
So as a rule of thumb, you may say bad control or covariate is bad control, if it might equally well be a dependent variable.
Right, if we could also think of it as a dependent variable to be explained as a result of the experiment or the intervention or the treatment.
Good controls, on the other hand, are variables that we can think of as having been fixed at the time the regress of interest was determined. So, either at this time, or already predetermined, so they should predate the outcome variable.
A control variable which predates the outcome variable is usually a good control.
So, when we look at some type of treatment effect setting, then a good control should be fixed already at the time of treatment.
So that precedes the effect, so that it cannot possibly have been affected by the treatment.
The bad control, on the other hand, would have been subject or would have been affected by the treatment and therefore distorts the regression results.
Actually, it would lead to something which is similar to a selection bias, as we will see now in an example which I give you on this bad control issue.
So, let's suppose again that we are interested in the effects of a college degree on earnings, on majors, for instance.
So, we simplify the setting here very much and assume that people can just work in two different types of occupations, namely either white collar or blue collar jobs.
And the white collar jobs are better paid.
So, the question is, should occupation be seen as an omitted variable in a regression of earnings on education?
And so, it may be that you ask the question where I want to explain earnings.
I want to find out what the effect of education on earnings is.
But I know earnings also depend on the type of job that people work in.
And there are white collar jobs and there are blue collar jobs and I know white collar jobs are paid better.
So, should I not include the type of job people work in in my regression to capture this type of effect which comes from the type of occupation on earnings, right?
At least if you are inexperienced, it could well be that people argue these type of questions.
And now that I have exposed the problem to you, perhaps you sense already that this is perhaps not such a good idea because the occupation white collar and blue collar can of course also be a result of the education
whose effect we actually want to estimate, right?
So, college degree probably opens the door to white collar jobs much wider than not having a college degree.
So, occupation is not only correlated with the pay, with a dependent variable, with the earnings, but it is also correlated with the treatment variable with college education.
This is where the problem comes from.
Note that some people of course have white collar jobs even without college and some college graduates may actually work in blue collar jobs.
So, we have to consider all possible cases here and we'll come to them, demonstrate them systematically to you on the next slide.
Suppose now we are worried about selection bias because as I have just told you, the blue collars and the white collar jobs are also a consequence of having received education, of having been or not having been on college.
So, you could again come up and say, well, can't we cure this problem by just looking at the earnings of a particular subgroup of our sample, for instance, just look at the earnings of people within a certain occupational group.
For instance, we just look at the earnings of people who are in white collar jobs and then look at what is the difference between those people in white collar jobs,
which have had a college education and those people with white collar jobs, which have not had a college education. Would this lead us to a good estimate of the effects of education on earnings?
Well, I think this sounds quite tempting to do it that way. Just look at one subgroup, either just the white collar workers or just at the blue collar workers.
But I would like to convince you that this is not a good idea, right? Even if we assume that college degree completion is completely randomly assigned, this would not any longer lead to an appropriate comparison.
So it would not be what people call an apples to apples comparison, but rather an apples to oranges comparison. And it is important that you understand why this is the case.
So let us look at the effect of college on college education on wages earned after completing college or not having completed or not having attended college for just those within a specific occupation.
So for instance, just at people who have white collar jobs.
How would we handle this problem? Well, we define a variable CI for college, right? CI is equal to one if individual I has a college degree and CI is equal to zero if the guy doesn't have college degree, either never attended or didn't complete successful.
And we also construct a dummy variable, WI for the work status, so for the occupational status of individual I and we would say that WI is equal to one for individuals which work in a white collar job, and it is zero for people who work in a blue collar job.
This is the reason why I simplified this example to the extent that I just distinguish two different types of occupations, which is of course very, very crude, but then we can work with just two dummy variables.
We just have four possible combinations of the dummy variables, both of them can be just zero one. Reality is much more complex, but the main problem you will sense very easily in this setting here with just two different types of occupations.
So, WI, same actually for CI, but here for WI is of course the observed occupation status.
Now the realization of this WI is determined, of course, by college graduation and by the potential outcomes of this variable given college attendance or no college attendance.
So we may distinguish two potential outcomes, we may say there is a W1I, which would be the potential outcome for individual I in case individual I attends college, not only attends but completes college, gets a college degree.
And W0I would be the potential outcome in the case that individual I does not obtain a college degree. So this is what the occupation status for individual I would be with college degree, and what the occupation status for individual I would be without college degree.
The observed value WI is then of course like we always do the result of combining the two potential outcomes with the two possible outcomes of the college variable, so of the treatment variable.
So either there is college degree CI is equal to one, or it is not the case. If CI is equal to one, then clearly the second term here vanishes, one minus one is zero.
And we have WI is equal to W1I, the potential outcome of individual I in the case of college education.
And conversely, if CI is equal to zero, so no college education, then this term here vanishes, and here we have one times W0I, which is the potential outcome of the person in the case of no college degree.
Now, there may be different types of individuals in our sample. Right. So there may well be some individual which I call J for instance, which would work in a blue corner job, even with a college degree.
So it may well be that for some individuals we have W1J equal to W0J and in both cases, this is just zero. So even after, even if individual J completes college and earns a degree there, you would go and work in a blue color job, and you would do the same thing if he doesn't attend college.
Obviously, there may also be people, individuals which are called individuals K, which go to a white color K job in any case, regardless of whether they have attended college or not. So both potential outcomes for W would be equal to one, completely regardless of what happens.
And then, of course, there may also be individuals which, well, this is a little bit perverse, but it may happen, right, individuals L, which I call them, which have as the result of, or as the potential outcome of college education, a blue color job, but as the potential outcome of no college education, they have a white color job.
Suppose, for instance, this is somebody who comes from a really poor family, which actually cannot afford college education, but the guy is really smart, right? So despite of not going to college, he would sooner or later qualify for a white color job, such people exist.
Now, suppose this guy wins a stipend and does go to college, even though his family is so poor, right? So we also need to ask what is his potential outcome in the case that he attends college.
Well, one possibility would be he receives a college education, but he attends classes in, I don't know, economics or philosophy or something like this, where he becomes fascinated by Karl Marx theories and voluntarily decides to go into a blue color job after graduation because he wants to show solidarity with blue color workers or he wants to prepare the communist revolution or whatever.
Something like that may happen. So even this type of individual is not completely out of the question. But what we would mostly expect to see are, of course, individuals M, which would have a white color job if they attend college and a blue color job if they do not attend college.
All right. Let's YI denote the observed earnings. For the YIs, of course, we can also write this in terms of potential outcomes. The observed outcome is the potential outcome, the potential earning in the case of college education, multiplied by the dummy variable, which indicates college education, plus one minus this variable times the potential outcome in the case of low college education.
So same structure as before.
Just for the sake of argument, not because this realistic, let us assume that the college degrees are just completely randomly assigned.
So we don't really consider decisions by our individuals to attend college and we don't really look at person's performance at college and some most of them probably get a college degree and others do not.
But rather, we now assume that college certificates to college degrees are just randomly given out to the population, basically without actually attending college. They just have proof of their college degree, even though they haven't been to college.
And then the question is what kind of job and what kind of earnings they get. So just for the statistically clarity here, we assume that the college degrees are randomly assigned, not because it's realistic.
In this case, the CI variable is independent of all potential outcomes, right, because we have just assigned them randomly. So CI is independent of outcomes in terms of earnings and outcomes in terms of occupations.
Under this artificial assumption, we have no trouble at all estimating the causal effect of the CI on either the earnings variable or on the occupational variable, because we have independence, essentially, right.
We have random assignments, so we have independence, which ensures that there is no selection bias. So we know the AT&T is equal to the AT&T is equal to the ATE.
So for instance, if we look at the earnings, we would know the expectation of earnings given college attendance minus the expectation of earnings, given that there's no college attendance, is the same thing as the expectation of the potential output in the, sorry, potential earnings in the case of college education
minus the expectation of the potential earnings in the case of no college education, right. And this is what we would like to know. This is the causal effect of college education.
And we could, in principle, do the same thing for the expectation, for the occupational status of the people. So we would have the expectation of WI, given college education minus expectation of WI, given no college education.
Due to random assignment, that would be just the difference of the expected values of the potential outcomes of the WI variable.
Any questions here for this example?
Apparently not.
Now, so we can estimate these average treatment effects by just regressing WI or WI on the college variable, without any covariates, right.
Again, assuming random assignment, but if this is true, then this in practice refers to this assumption that it is true, we can estimate the average treatment effects by just simple regressions on CI.
But now assume that somebody suggests to use the WI variables of the occupational status as a control in finding the causal effect of a college degree on earnings.
And as I say this, we will see this is a bad control because WI is affected by the treatment, right. But somebody comes up and says with the same arguments that I have already advanced a couple of minutes ago, WI also explains part of the earnings.
So why don't we condition on WI in our regression, which regresses earnings on college education and now as a control also on the WI variable.
Well, actually, there are two possibilities. We can regress on the WI variable, or we can say that we regress the earnings just on CI, but only for those observations, let's say, where workers are white collar workers.
So we just use a sub sample of white collar workers. That's the first possibility I've written down here, right. So in that case, we would control for occupational status by just using the sub sample of white collar workers and then regressing WI on CI.
We know that regressing WI on CI for the full sample is fine, but this gives us the correct effect. Now somebody comes up and says, well, why don't we just do it for the white collar workers.
I want to convince you that this is not a good idea. Or we use WI as an additional regressor in the regression of WI on CI, so we condition in the regression on WI, use the full sample and have WI as second dummy variable in the regression.
So these two possibilities exist.
The problem is that the WI, as I said, is bad control and that in both cases, the estimates would be distorted, would be biased and would lack the causal interpretation.
Let's look at the first case where we just use the sub sample. So we just look at the white collar workers, where we use those people which work, white collar, which have been to college and those white collar workers which have not been to college, which certainly also do exist.
So what would be the analysis which we do here? Well, formally, it is this difference we look at.
It is the expectation of WI given college degree and white collar job minus the expectation of earnings given no college degree and again white collar job.
So just the sub sample of people with white collar jobs. This is why we have WI equal to one in both terms here, which seems fair and balanced, right?
Now we can rewrite this difference here in the following form. We can say this is the same thing as Y1, either potential outcome in the case of college education because this is conditioned on CI equal to one, WI equal to one.
Same thing, but now I write W1I equal to one because this is the case where people have received college education. So it is W1I, but this WI is actually the same thing as W1I.
Minus the expectation of the potential outcome in the case of no college education, so potential earnings in the case of no college education.
Given that there is no college education and the WI variable would then of course also be the W0I variable because this is a white collar status despite of not having been to college, right?
These two equations are the same type, the same equations as on the previous slide, just for convenience. And here I have just written out what I already said.
You can read it at home. It's no new content, actually, just in order to understand this equality sign here, why we can identify this difference with that difference.
Now we had in the review of probability rule one, if X and Y are independent, then it is true that the conditional expectation of Y given X is equal to the unconditional expectation of Y.
We know we have joint independence of all those conditional outputs or all those conditional earnings and occupational, sorry, those potential earnings and potential occupation statuses, which are here, the curly brackets.
These are all independent of the college status because we have assumed that college degree has been randomly assigned, right?
So since this is the case, we can make use of rule one here and can say the difference between these conditional expectations, Y1I given these things, Y0I given these things, so precisely this line here, right? Same line.
This difference is the same thing as the expectation of Y1I given W1I is equal to one minus the expectation of Y0I given W0I is equal to one.
So we can kick out the conditioning variable CI here and there because our potential outputs are independent of CI, right? Using rule one. So we arrive at this difference here.
Now, this shows now why we have not a fair comparison here, why it isn't apples to oranges comparisons and which we do in this bad control problem because the difference we had on the last slide was this difference here.
Both people work white collar, WI is equal to one, right? But this is the expectation taken with reference to people who have a college education and work white collar and this is taken with respect to people who have no college education and work white collar.
So we immediately see that there is a selection bias in this exercise here because we can add a term here or subtract and add the same term to write the same difference in this way.
So here I look at the expectation of Y1I, I get it here given W1I is equal to one. So this is this term here minus, which is the new term, Y0I given W1I is equal to one.
And since I have subtracted this expectation of Y0I given W1I is equal to one, I have to add it here, which I do here plus the same expectation, Y0I given W1I is equal to one.
And then I subtract this thing here, which is this term.
And now I see this thing is the causal effect of a college degree, right? It's the difference in potential outputs given that the expectation, the occupational status is W1I equal to one. So that's a causal effect, which we have here.
But this here is a selection bias because it's the difference between potential outputs Y0I, in both cases Y0I, between two different groups, namely those people which work white collar after having received college education and those people which work white collar without college education.
So this is the selection bias here.
So we see that we actually have taken averages over different groups of people, those which would have a white collar job if they had a college degree against those which would have a white collar job, even if they had no college degree.
And since these are not the same groups of people, we have a sample selection bias here.
Or, in other words, the observed difference in earnings between those with and without a college degree, conditional on working, no white collar job as we have done here, equals the causal effect of college on those people who work at white collar jobs when they have a college degree plus selection bias,
which reflects the fact that average earnings without a college degree are probably different for people who would be white collar with degree and people who would be white collar even without degree.
The size of the selection bias is unknown to us. It can be positive or negative.
It depends on the relation between the occupational choice, the college attendance and the potential earnings. So we won't go further into this.
But even if there were no causal effect of education on earnings, so if it were the case that Y1i is equal to Y0i for all i, we would not recognize this if we condition on occupation status, but rather we would find the selection bias and mistakenly interpret it to be the causal effect.
Unless we understand the theory. So if this was perhaps a little too fast, suppose that actually there is no causal effect of college degree at all. Suppose that these two potential outputs are the same.
And this effect here would be zero. But in our inference, where we condition just on the sub sample of people who work in a white collar job, we would find this effect here.
And unless we understand the theory would think that this, what is actually the selection bias, that this is the causal effect of college education. So we would be completely led astray and get a completely false idea of the effect of college education, which truly under this rather unusual assumption would be zero.
But we would find some probably nonzero effect for that.
So it's really important to think thoroughly about this question and about the effect which the bad control would have.
Now this concludes the first case, the second case where we use WI as a regressor, where we condition on that in the regression, I will treat in the next lecture, which will be next here.
As I told you, now we will move to interaction. For those of you who do not want to go to interaction, I already wish you a very blessed Christmas and a good and healthy new year.
As I say, first lectures of the next year will be recorded lectures. So we have interaction again, then the second week of the semester and the new year.
And those which are interested in the interactive meeting, please now log out of this meeting here and log into the interactive mode under the link which I sent to you yesterday.