Thank you.
Welcome to today's lecture on estimation and inference. Are there any questions. Please raise your hand, if there are.
Let's see any questions, and I would like to review a little bit what we did in the last lectures, which is why I went back my slides to the late theorem.
Just to remind you what it says since we are now going to strengthen the late theorem with the bloom result which I already prepared for and mentioned in last lecture.
The topic I was talking about in the last lectures is the problem that we may have a study, which aimed at valid randomization of the treatment and control group.
But that randomization was essentially made ineffective by endogenous decisions taken in particular in the treatment group because people in the treatment group may have an option to still decide whether they want to take treatment or not.
So they are assigned to receive treatment by the researchers who design the study, but it is still up to those people who have been assigned treatment to accept treatment.
And if they do not accept treatment for whatever reason, this may be. This introduces a self selection mechanism in the research design, which is very likely to destroy the effect of randomization, because probably those people who
endogenously decide to have treatment versus those which decide not to have treatment.
These two groups are then not random selections anymore but there are certain motives of some people to receive treatment which may depend actually quite likely on the expected outcomes of treatment.
Similarly, those people in the treatment group who do not want to have the treatment may also take their decision with respect to those outcomes which they expect.
So if the treatment has effect heterogeneity, so if we must expect that the treatment will not benefit everybody just in the same way, then it is likely that those people expect less effect from treatment than others have a higher inclination to refuse treatment.
And therefore, if we can just look at those people which actually received treatment which actually were treated, we are likely to overestimate the true average treatment effects.
So with this in mind, we defined four different groups of people, the always takers, the never takers, the compliers and the defiers, where however we excluded the defiers right away by the monotonicity assumption because many contexts,
the defying behavior is rather weird behavior as it would say that people would take treatment if they are not offered treatment but they would refuse treatment if they are offered treatment.
So that may be weird behavior and we have excluded it here by virtue of the monotonicity assumption and if this monotonicity assumption holds and we also have the independence assumption between the instruments Z and Z on the one hand and then the treatments and the potential outcomes on the other hand,
then we have this late theorem which essentially states that by using the instrument Z in an instrumental variables estimation technique, we would identify the treatment effect on the compliers, but only on the compliers and usually this would not be the treatment effect in the sense of the average treatment effect.
However, there are special circumstances when we can actually expect the late to equal the ATT and this is the case when we do not have any always takers.
That's the result of Bloom which I explained already last time and we're going to prove it now, but before doing so I would like to go back to the example I have already discussed at the end of last lecture, namely the job training partnership act which essentially was an act of Congress, which enabled people who have problems
of finding a job in the labor market to receive treatment in the sense of job training.
And there was a randomized evaluation of this quite costly job training partnership.
Act of training program, where people were randomly selected to be offered this type of training, and it turned out that only 60% of those assigned to training opted for training and then of course receive the train.
Whereas in the control group, there were 2% of people so rather minor share who somehow managed to get the training anyway, even though they were not offered the training.
We have a small fraction of always takers in our study but it is as I say, just a small fraction and since we have chosen the treatment group randomly would expect that we also have 2% always takers in the treatment group.
Now I would like to ask you, hoping that you have thought through this example over the last days.
When we have the treatment group here and 60% of those assigned to treatment actually opted for treatment we have those 40% which did not take treatment by their own decision.
What kind of people are these 40% can you please raise your hand if you know the answer those 40% in the categories which we established for the analysis of this type of problem with effect heterogeneity.
How would we call the people, those 40% which did not want want to take treatment.
Can anybody answer this question.
It is not such a difficult question. I would hope that some of you are able to answer the question those 40% or what we call them.
Be aware that might be a question which I might pose in the final exam.
Okay, somebody has an answer.
Never take us. That's the suggested answer the never take us certainly those which would refuse to take treatment if they are offered treatment and to that extent, the answer is not incorrect to that extent is correct.
But of course there may be also other people would not take the treatment and which people would that be
another. Yeah, there is. Okay, there's the second answer the never take us and the fires. Yes, and this is what I wanted to emphasize and what you should make clear to yourself.
Those people who refuse treatment are not necessarily all coming from the same same group of people. They may either be never take us, or they may be the fires.
As long as we have no reason to impose the monotonicity assumption, we must account for the possibility that there are also the fires, even though as I said it is perhaps weird behavior and unlikely behavior just but just in principle, those people who refuse treatment
can either be never take us, or they can be the fires. Right. We have those four groups of people and it's clear they always take us will always take the treatment and the compliance apparently do take the treatment because they have been offered the treatment.
So, these are out as candidates for for the answer of my to my question, but clearly both never take us and be fired fires could be at the root of those 40% who refused to take treatment or decided not to take treatment.
All right, this is just to make this clear again. And then let me move on with the example which I have already played.
So I'm going to you there are three different ways to estimate things here either just least grass estimation where we regress the post program earnings by ISO outcomes on the treatment status, or the reduced form estimates where we would regress why I on the instrument
here would be that treatment has been offered, rather than that people have actually been treated. And then there's the third estimation strategy that we would regress the outcomes on the treatment status.
So on the variable indicating whether people have actually been treated have actually received training, but instrumenting then the treatment variable with the instrument, the variable which indicates where the treatment offered to the people.
And we know from the late theory which I just reviewed that this IV estimate would yield as the estimate of the late effect so the local average treatment.
And I have then discussed with you this table here, where we have the three estimation strategies, just regressing on treatment status would be columns one or two so without with covariates regressing on the instrument would be columns three and four without
and with covariates so there are always two variants of estimates here and this would be the instrument variables estimates that we saw as expected treatment effects are much greater in the invalid estimation technique where we just redress on treatment status,
and in those other four estimation strategies which are actually valid estimations where the treatment effects are much smaller and actually for men often not even significant actually now larger for women than for men, whereas it seemed to be the opposite in the simple OLS regression.
So, and that I have essentially said, and I just mentioned again that the self selection in the treatment group here destroys the random assignments assignment, unless which is a very unlikely, unless the decision to comply.
So the decision to accept treatment to enroll in training program was itself independent of potential outcomes, but this, as I say, is not so very likely because we should expect actually that people who hope to benefit much from from the training have a greater inclination
to enroll in such a program that there would be no anymore.
All right.
I then start last lecture with the term intention to treat effect and actually encourage you to read the slides yourself, once again but I think, or now think that I should perhaps explain this again with the intention to treat effect because this new terminology.
We have in the table the reduced firm estimations in columns three and four and reduced form means here that the outcomes were regressed on just the instrument, so just on the variable which intended which which expresses that a certain group of people
was intended to be treated by those people who designed the study. They assigned certain people to treatment, and later it turned out that 40% of those assigned to treatment did not want to be treated, but all of them were intended to be treated.
And this is why we say that the estimation results of columns three and also without with covariates identify the intention to treat.
This is a perfectly valid result, because the people are randomly assigned to treatment. So the zi variable is actually assigned to individuals.
The question is more whether this is very informative what we get when we identify tension to treat.
But for say cost benefit analysis may still have some value, because the costs may depend on the number of people which we assign to treatment since those people which do not enroll in treatment in the training programs may nevertheless cause costs,
or there may actually people who decide on not completing the training, only after they have enrolled and then perhaps capacities have been financed for those people that some instructions may actually have already been given to these, these, these people,
but still they do not complete the training and therefore the funds on them have been wasted.
So for such policy considerations, it may also be interesting to identify the intention to treat effect.
Obviously, the ITT effect should be small relative to the average cause and effect on those which were in fact treated.
Because the treated of course benefit whereas the 40% who were intended to be treated but have not been treated have typically no or just a very small effect. So the ITT effect should be much smaller than the average cause effect.
And we see this in the table because the reduced form estimates are much smaller than in the other estimation strategy.
I will go back to the table here where you see this is, for instance, without covariates the ITT for men, some $1,100, whereas the effect on the compliers identified by IV is 1,800 so quite a bit larger than it is for the ITT effect.
The same is true for the woman if you compare the $1,200 here to the almost $2,000, which is the treatment effect in the case of the compliers.
Okay, the columns five and six, then give us late identified by instrumental variables as you know.
And here we know now that the late effect is approximately equal also to the ATT, so to the average treatment effect on treated because the number of always takers is very small, just 2%. So we may say, okay, we approximate the 2% with 0%.
If there were 0% always takers, then the late is equal to the ATT, which is the LUM result, which we are going to prove in a minute.
Again, results for men are only marginally significant and smaller than for women this I have said, even though the difference between effects for men and women is not significant, I think that I have not yet emphasized.
Now, make a check, please. The IV estimates are equal to the reduced form estimates divided by the estimated coefficients on the first stage regression.
You can actually not see this directly in the table, but you know that it should be the case from the theoretical analysis that we have made already.
I think before Christmas it was, I showed you that the IV estimate beta one hat is equal to the reduced form estimate gamma one hat divided by the estimate of the first stage regression theta one hat.
And if we want to verify this result, which we received in our theoretical deliberations, we want to verify this on this example here, then we just have to exploit a little bit of information which is not directly visible in the table.
And perhaps you can just think for a couple of seconds whether you know which information we should use for that. So if you recall that the first stage regression is the regression which regresses the treatment variable D on the instrument variable Z, yielding theta one hat.
Just to recall what the notation was, running regression of D on the instrument variable Z was the first stage regression at the unit theta one hat.
What do you think will be approximately the estimated first stage regression coefficient theta one hat? Do you have any idea on how you could come up with a number now to tell me what this theta one hat is approximately?
Theta one hat is probably a liking to be.
Try to make it as intuitive as possible for you when you regress D on the instrument Z, then you regress those people who have actually received treatment on people who were offered treatment.
So what do we know about the relationship between the number of people who have been treated and the number of people who have been offered treatment and what kind of infer them on the coefficient of the Z variable in the first stage regression?
Yes, there's an answer.
About 60% exactly, because we know that about 60% of those which have been assigned to treatment have actually opted for treatment. So this is the correct answer. Thank you very much.
We know that there are about 60% of those intended to receive treatment which actually wanted to have treatment.
So, even though I didn't give you this now formally this result, we should expect that the first stage regression coefficient data one hat is approximately equal to 0.6.
We do not have the data now, at least we haven't worked with them now, but we can verify this indirectly by looking at the regression results in the table.
Because what we know is the old result from before Christmas, a beta one hat, the IV estimate is equal to gamma one hat, the reduced form estimate divided by the first stage estimate data one hat.
So rearranging that we know that data one hat is equal to gamma one hat divided by beta one hat IV.
And these two results we have in the table, right column three gave us the reduced form estimate the gamma one hat and column five for instance if we just consider the case without covariates gave us the beta one IV estimate.
So when we divide now the estimates in column three of the table by the estimates in column five of the table, we get for men, the relationship of 1117 divided by 1825.
This is 0.61. And for women we get 1243 divided by 1942.
This is 0.64. So in both cases, it is approximately equal to the share of 60% of people who accepted the treatment, so who were the compliers in our case.
You can do in principle the same thing also for columns form six in which we have covariates in the regression.
Theoretically, actually, it is not true anymore that with covariates beta one hat IV would be exactly equal to gamma one hat divided by data one hat.
So in general, this will not be equal, but often it is the case that is still approximately the case and the reason for this result is again of course randomization.
So the covariates should actually do orthogonal to the treatment variable and the sorry to the assignment a variable so to the instrument.
Now, in our setting here we know that we have almost no always takers. So the treated population consists predominantly of the players just treated population so just those 60% who actually opted for for treatment, they are predominantly compliers.
We know there are about 2% of the grid, which are always takers, but that's a small share. So predominantly these are compliers and therefore the estimates in columns five and six are close to the estimates of the ATT.
And that is the bloom result from 1984 that we can use the instrumental variables estimation and technique actually in order to also identify the ATT.
If there are no always takers, or perhaps in a somewhat more technical wording. If we have a randomized trial with one sided, not compliance.
Yeah, that is perhaps rather technical not so intuitive so it is perhaps easier for you to memorize that is memorize the bloom result is saying the instrument variables technique will not only identify late.
But the late will also be identically identical to the average treatment effect on treated. If we have no always takers. That's probably easier to memorize.
And here is the bloom result as a proposition. So suppose we have the same assumptions as the late theory, so particular we have the monotonicity assumption and we have the independence assumption.
And additionally, we do not have any always takers. What would always takers be formally will always takers are actually those people who take treatment, even if the instrument is switched off.
So if the instrument assigns them to the control group to the non non no treatment group. Right.
So if it is true that the expectation of the treatment variable, given that people are assigned to the control group is zero, then this means there are no people in the control group who take treatment.
That is to say, there are no always takers. So formally the assumption of no always takers is the assumption that the conditional expectation of Di, given that is equal to zero, that this expectation is zero.
Now in this case we can strengthen the late result to indicate that the instrumental variables estimator will converge actually to the average treatment effect of the treated.
So that's expressed by this formula here. The plim of the instrumental variance estimator beta one hat is as we know, well, I write here the reduced form estimate gamma one hat, actually you should delete perhaps
the heads or should write here better. I will improve on that. I should write here is the plim of the reduced form estimate gamma one act because the expectations here are the difference between these conditional expectations are already true
parameters and not estimates any anymore. But I wanted to make it clear that we have the beta one hat here and then the usual formula beta one hat IV is equal to gamma one hat divided by delta one hat.
So I forgot to write plim here and there and I will correct this after the lecture.
But in principle, you should see what we have in this ratio here in the numerator. That is the plim of gamma one hat. So this is gamma one. Why is this so when you see this is the expectation of the outcome conditional on the instrument switched on minus the expectation of the outcome conditional on the instrument switched off.
So that is very clearly the coefficient of the reduced form regression and that we have denoted by gamma one.
And here in the numerator in the denominator of this ratio, we have the expectation of treatment, given that the instrument is switched on minus the expectation of treatment given that the instrument is switched off.
So here we have the first stage coefficient data one or the plim of the first stage estimates data one hat.
Right. So that's the usual result, which we know from instrumental variables analysis or nothing new here. The plim result is this equality sign here that this ratio on the left hand side of this equation is equal to the ATT.
So we have perhaps I should perhaps explain that this expression here is the ATT, where we have a conditional expectation again that namely conditional on people being treated the is equal to one.
Right. And then we look at the difference between the potential outcomes for those people who have been treated on the expectation of this difference. Well, this is of course the ATT.
So the plim proposition says that the experimental variables estimator beta one hat is a consistent estimator of the ATT if it is true that there are no always tickers.
That is a little technical.
I have not taken some pains to explain it as well as I think I can. I'm happy to explain again if you have questions and it seems there is a question.
Oh, the question is, can we use this result when the percentage of always tickers is small. Yes, but only approximately that. Right. That is basically just sort of the
an approximation to the plim result. Right. If you have a small, very small percentage of always ticker then you can say well it is likely that the plim result will hold approximately.
And this is precisely what I have done in discussing the job training partnership act where the percentage of always tickers were 2% and then I have said 2% is small. Right.
Of course, the question can always be what exactly is small and how strong will be then the difference between the ATT and the late effect there will be some difference but hopefully if this percentage of always tickers is small, then the difference is also.
Very clearly we work here just in the world where we use approximations, because the whole result, the brutal result is an asymptotic result. So the plim of beta one hat is equal to the ATT in finite samples and we always just have finite samples.
It is clear that even this thing here will not hold exactly or that this will not even hold exactly if the number of always tickers is truly zero.
It will not hold exactly in any finite sample. Only when the number of observations converged to infinity would we have this result. So whatever we do, it is always just an approximation.
No, I think there was no new question. Now, let's go to the proof if there is no other question on that.
No wait. For some reason I still see a question here but I have answered that one.
So it doesn't disappear. Anyway, let's prove this result.
So we start out with the late theory and the late theorem told us that the plim of beta one IV is equal to the effect on the compliers.
Right. So it's the expectation of the difference between the potential outcomes, given that we just look at those people which change their behavior.
When the assignment to treatment changes. I remember this notation D zero I was the status of treatment when the instrument is switched off.
So, let's say when people are not assigned for treatment, and the one I is the treatment status when the instrument is switched on so when people are assigned to treatment.
So those people who change their behavior due to having been assigned to treatment for these people we look at the difference in expected outcomes and then we have the late effect here.
Now in the bloom theorem in the blue proposition, we have to exploit the additional condition that the expectation of treatment.
Given that the instrument is switched off is equal to zero.
And how can we exploit this. Well we start with the right hand side here, zero is equal to the expectation of the treatment instead of status given that this is that the instrument is switched off.
And then we can replace the treatment status DI by the potential treatments the zero I plus difference between the one I and zero I multiplied by treatment status, supposing again.
Sorry about the instrument status supposing again that the instrument is just binary either zero or one.
And here we condition on the instrument being equal to zero. So, very clearly, this term here, just cancels, and therefore, this is the same thing as the expectation of the zero I even that that is equal to zero.
And after this kind of trivial rearrangement we exploit the independence assumption, we know that the zero I shall be independent of the instrument.
The conditional expectations here is equal to the unconditional expectation. So we know that the expectation of the zero I is zero, the unconditional expectation of zero I is equal to zero.
All right.
The treatment variable and the potential treatments can only assume two values, the zero I can either be one, or it can be zero.
So, when the expectation of the zero I is just equal to zero, then this applies that the zero I is actually zero with probability one.
So the probability for the zero I being equal to zero is apparently equal to one.
We know the probability that the one I is greater than zero I is just the probability of the one I being equal to one, because the zero I is always equal to zero.
Right. So whenever the one I is greater is that whenever the one I is equal to one, it is also greater than to zero, since the zero is always zero.
So, we know the probability that the one I is greater than zero I is equal to the probability of the one I be equal to one.
And note that this event here was the event which we looked at when we looked at the compliers.
Now, what we also know is that the probability of the I being equal to one is the same as the probability of the one I being equal to one.
So this probability here, the one I is equal to one, this probability here, that's the same thing as the probability of the I being equal to one.
Why is that? Well, because we know the probability of the zero I be equal to zero is again one.
Right. So, very clearly, the treatment variable assumes value one with the same probability as the potential treatment variable assumes value one.
Here we have so far just exploited what is implied by the additional assumption.
This way I want to go that the expectation of the I given that I is zero is actually equal to zero.
Right. We have derived now from this assumption these properties here and here and can use them in the analysis of what beta one IV converges to.
Oh, there's a plim missing here too.
It should be plim beta one IV here is equal to the expectation of Y one I minus Y zero I given that treatment status changes.
Right.
So, this is the late result that we have here.
Now we can make use of the fact that we know the unconditional expectation, which is E of Y one I minus Y zero I the unconditional expectations always equal to the conditional expectation times the probability of the conditioning event.
So times probability of the one I correct and the zero.
This case I have written this in the opposite way by saying the marginal. Sorry, the conditional expectation is equal to the unconditional expectation divided by this problem.
Now we know the probability of the one I greater than zero I is equal to the probability of the I being equal to one by using these relationships here.
Well, if we have this, then again we have an unconditional probability and unconditional expectation divided by a probability. So we can write this as a conditional expectation, which is the expectation of Y one I minus Y zero I, given that people have been treated.
And that means we have the entity. Right. So this is the expectation of the difference in potential outcomes for all those which actually have been treated.
And that proves the blue result.
Any questions here.
So that is not the case.
Then let me conclude with a couple of remarks on compliance, basically.
The remarks now are more general than the setup, which I have discussed. So, are. And then the final exam I will not give you the more complicated setups but just to give you some perspective on the late and on the bloom result, I
conclude now with a few remarks on more general on the more general setup in which the later result may be used.
The thing that we have not done so far is that we looked at more than just one experimental variable. Right. But that may easily be the case in empirical settings that we have several instrumental variables, and then maybe the case quite often actually
that each instrumental variables, each instrumental variable identifies a unique causal parameter.
So, in our setting we just had one instrumental variable and we have identified one causal relationship, which is the late, but if we have more than one instrumental variables, then maybe the case that each instrumental variable identifies a unique causal parameter,
which is then always specific to a certain subset of the compliance or subpopulation of the compliance, namely the subset of compliance, which is specific to this particular instrument.
If we have effects heterogeneity, then it is clear that different valid instruments for the same causal relation may result in different causal effects, so in different estimates, simply because we have effects of heterogeneities.
This would then imply that the differences between the compliant subpopulations, so between the subsets among the compliance, which are specific to certain and different instruments, that these subpopulations have different treatment effects,
which may be interesting in its own right to see that the compliance are themselves not a homogeneous body, but that they are made up of smaller groups of people, each of which have their own treatment effects, which differ from the treatment effects of others.
So effect heterogeneity can go very long ways up to the situation where each individual actually has its own individual treatment effect. There is complete effect heterogeneity.
In such settings, it is of course important to know the compliance for different instruments, so we could do a reasonable analysis only if we have reason to believe that we can identify which people respond to which type of instrument.
If there are two instruments, which affect essentially the same group of people and for the same reason, then we would expect both IV estimates using the two types of instruments to be similar.
If the compliant subpopulations associated with two or more instruments are very different, but the IV estimates are similar, then of course we can conclude that the treatment effects are perhaps homogeneous, so that we do not have effect heterogeneity as we have initially supposed.
I see there's a second question. May I still ask you to pose your questions via the chat rather than the Q&A, please? Because in the Q&A, this indication of questions just go away after I have answered the question.
Now, somebody asks one question about the syllabus. Do you think we can do the last chapter panel data? Thank you.
Quick response, no, we cannot do panel data. I won't have the time for that. But also my request, while I'm in the middle of the lecture, please pose questions which really refer to what I just say in the lecture and the more general questions, you have opportunity either at the end of the lecture or at the beginning of the lecture.
But please don't do it right in the middle of something else.
Okay. In general, IV is very helpful, a powerful tool. We have experimental studies where we have full control of our data set because it would help us to analyze failures in randomization.
But it is also a powerful tool in observational studies where we do not have full control because it helps us to solve the problem of limited variables, as we have discussed this, or of measurement errors in the independent, so in the explanatory variables.
But there is, of course, always the problem that we can use instrumental variables techniques only if we really have suitable instruments.
And that is, in practice, often difficult matter to find a variable which fulfills the exclusion and validity restriction. Actually, the exclusion and validity is restricted, it's the same thing.
That's why I write exclusion and validity restriction. So it's just one thing I have initially introduced in this lecture as validity restriction and later mentioned it as exclusion restriction.
I just would like to add so that you are not confused by the two terms here. You find both in the literature but exclusion restriction and validity restriction is the same thing.
It is the restriction that the instrument does not vary with the error term of the endogenous variable so that the instrument does not affect the endogenous variable directly but only through the regressor which the instrument is replacing in the instrumental variables estimation.
This is it for the long chapter on instrumental variables techniques, and you've seen there are many different applications, many different settings in which we can usefully employ instrumental variables techniques.
I hope that you have understood this well and learned how to use instrumental variables techniques, and if there are still questions concerning IV, then please post them now before I move on to regression discontinuity setups.
Yes, I see there's an indication of a question, please write your question in the chat.
Does anybody else have a question when the first one is still coming in? Yeah, here it is. Let's see.
Can you please explain again in the first sentence? I slide 55. Why?
Sorry, this is a little, so on slide 55, you want me to explain something, the first sentence.
Perhaps you can make your question a little bit more precise, but I move back to slide 55, which is this here.
Where I explain to you that potential outcomes could now in principle depend on both the values taken by zi and di, and there's more explanation I hope coming.
Not yet completed. Why both z and d?
Okay, I think I understand the question, probably. Let's see.
Well, what we have here is that we have, let's say, a correlation, an imperfect correlation between instrument and regressor.
So between treatment variable di and the instrument variable.
It is not the case that the treatment status and the instrument variable are correlated one-to-one so that they convey the same type of information.
But rather they convey different types of information and actually it may be that the instrument status signals something like a person has been assigned to treatment.
But the treatment status does not signal the same type of information or does not conform with our expectation that somebody who has been assigned to treatment is actually treated.
But rather the treatment variable indicates that the person which has been assigned to treatment did reject treatment, did not take advantage of treatment.
So we have more information now on potential outcomes or actually we could think that we have more information on when certain outcomes emerge, namely not only whether treatment has been taken or not, but also whether treatment has been offered or not.
So therefore, one might think that the outcomes depend on the complete set of information which we have, that the potential outcomes depend on whether treatment has been actually taken and whether treatment has been offered.
So it could be that or you might think that in a causal regression, you would now have to regress the observed outcomes yi on the di and on the zi using the instruments as a second regressor rather than as an instrument.
But if the exclusion restriction is valid, then we know that the instrument affects the outcome only through the treatment status, only through the variable di.
And if this is the case, there is no need to use the zi variable as an independent regressor along with di, but rather the zi then helps us to instrument the di and therefore solve the problem of di's endogeneity.
So the first sentence, and I hope now this answers your question, emphasizes that in principle, one could suspect that zi is now a useful technique regressor.
But then, as I have said here in the sentences below, in this case that I wouldn't be a valid instrument, it would violate the exclusion restriction.
And therefore, by assuming that i is an instrument for di, I have essentially already assumed that exactly this here is not the case, the potential outcomes do not depend on zi, they still just depend on the di.
And this is why I can continue to use this notation y0i and y1i.
Does this answer your question?
All right, so thank you, so I suppose it does.
All right, any other questions?
I don't see any, so let me close slides seven and we move on to slides eight, which deal with regression discontinuity.
Here is again what you can read to accompany what I've said in the lecture, again from the two books by Angrist and Bischke, chapter six in the Mostly Harmless Econometrics, and chapter four in the Mastering Metrics, whatever you prefer, or perhaps you want to read both.
So what is the general idea of regression discontinuity or sometimes called regression discontinuity design?
Well it's actually a fairly easy and intuitive idea, namely that there are certain settings where a specific threshold discriminates between people who receive treatment and people which do not receive treatment.
And this threshold may be something which is exogenously and arbitrarily set by somebody.
And then the idea of regression discontinuity is that we compare people just above this threshold and just below this threshold.
Basically, regression discontinuity design is a design which we use when we have huge numbers of observations, which is often the case when we study, say, labor market data.
For instance, if you have certain programs by the government which encourage people to assume, let's say, additional work by giving certain incentives like tax reductions or something like this,
then it may be that millions of people actually respond to this type of incentives and we may have millions of data.
Now it may be that some people are eligible, let's say millions of people are eligible and millions are not eligible, depending on a certain threshold.
For instance, only people who work at least 90% of their time already with regular contribution to social security systems are allowed to take on second jobs with reduced payment obligations for the social security systems.
Labor market programs like these do exist, right? So you must work at least 90% of your time already. And then you have people who work just less than 90% and other people who work a little more than 90%.
And the letter are entitled to respond to the incentives set by government programs and to take up the type of second jobs with less requirements on social security contributions, whereas the former who do not have at least 90% of their labor time devoted to some employment with social security contributions are not eligible.
So you may have even close to this threshold of 90%, many, many people who are just a little bit below and many, many people who are just a little bit above the threshold.
And if this is the case, then it is often argued that these people who are just below or just above the threshold are likely to be quite similar.
That is actually sort of a random division of a larger group of people who are very similar when this 90% threshold is exogenous and arbitrarily set by the government, right?
So that we would be in a position to just compare people which are below this or to compare people which are just below the threshold with people who are just above the threshold, assuming that these people are really very similar and thereby being able to identify the treatment.
I have a number of examples for you here, I think three examples, which are along the same lines as this one example I already gave. The first one is indeed a labor market example but different setup.
In this case, it is about unemployment.
It is often the case that unemployment benefits are paid out by the social security systems of a government for a specific duration of unemployment.
For instance, for six months. That's the case in German, right? After six months, unemployment benefits cease and you go to some other kind of social support with lesser support.
So unemployment benefits as a percentage of your formerly earned income you just get for six months and after six months, you drop down to basic social support which can be substantially less if your previous salary and therefore your unemployment benefits have been high.
Which means that we may study the behavior of unemployed, which I have so far still a duration of unemployment, a little less than six months, with people who are unemployed for more than six months.
They may, for instance, look at their labor market behavior and what kind of effort they put in getting a new job if they are still on rather high social support or if they are now on rather low social support.
So a different example or another application of regression continuity could be another threshold which in German law we call the Jarvis Arweitz and Gelt-Grense.
So the Jarvis Arweitz and Gelt-Grense is a certain level of income which you must have in order to be free of the compulsory public health insurance.
People who are below the Jarvis Arweitz and Gelt-Grense, so you have rather lower income, are required to take public health insurance.
But above the threshold, this is voluntary and people can choose not to take public health insurance at all, go to private health insurance or not to be insured at all because the government says what people who have a certain amount of income high enough would be able to cover their medical costs even in the case in which they are not insured.
So above the Jarvis Arweitz and Gelt-Grense, you are not obliged anymore to take public insurance.
Or a third example, university admission for some fields of study in Germany or German universities is based on the school grade point average, what we call the Abiturf.
So the average grade you receive in your final exam or our school type.
Therefore, the probability to attend a given program like medical school, for instance, at a given university changes discontinuously at a specific grade point average threshold, which we call the more clauses.
So if your grade is good enough, then you satisfy these type of more clauses requirements and then your probability of being admitted to the field of study, which for some reason is restricted, medical studies, your probability is much higher than if you are below the most clauses, so your grades are not good enough.
Typically, there are also ways to enter medical school if you have not met the most clauses, so if your grades are worse than what is required so that the probability is still positive, but it is not as high as the probability was or as the probability would be if you met the most clauses requirements.
So the core idea of regression discontinuity is actually that the probability of treatment or this case of being accepted to a certain field of study changes discontinuously at a specific threshold for a continuous variable which is related to the treatment.
Which means that we have something like an exogenous shock in the treatment or exogenous event treatment. Individuals above and below the treatment would be assumed to be the same, at least in a small neighborhood around this threshold, except that they differ in either meeting the threshold or not meeting the threshold,
and therefore also differ in the consequences of meeting or failing to meet the threshold.
In an RD, so regression discontinuity research design, we would just exploit the precise knowledge of this rule, which determines treatments of this arbitrary decision to assign treatment to those people which are above the threshold and refuse treatment to those people who are below the treatment.
Since we live in a world where many such rules exist, so we have a really rules based society, we have lots of such arbitrary rules and therefore the rules equip us with some type of quasi
or data which we can use to validate quasi experiments.
If this clarifies the general idea of regression discontinuity designs, since now I think where discontinuity comes from, we can in principle differentiate between two different types of RD, namely sharp RD and fuzzy RD.
Sharp RD is rather rare in economics, because sharp RD would say that the probability of treatment changes from zero to one, or vice versa, at the threshold.
So you would know below the threshold you have no chance of receiving treatment and above the threshold your chances are 100%.
That is rather rare. These kind of always takers and never takers would not occur actually if we had sharp RD, but in principle, of course, this is a possibility and we will analyze this possibility first because it is easier to analyze as fuzzy RD, which however is in real verb settings more common.
Fuzzy RD then says that the treatment assignment does not perfectly depend on the threshold.
So there are also other dimensions which have an influence.
Like what I just said about medical school being admitted to medical school, the numerous clauses is this important threshold. Typically, you have to have a very good grade in your avatar in order to meet this threshold.
It's typically 1.0, so just the best score you can possibly obtain in your avatar, this you have to meet in order to be admitted to medical school if you apply for medical school, but those people who have
other grades, grades not quite as good as 1.0 can go through certain tests or they can wait for a semester or two semesters or they can go to private school or they can go somewhere to another country and or start with a different subject that later reapply to medical school.
So there are also other possibilities of being admitted to medical school and meeting the numerous clauses requirement. So the probability that you are admitted to medical school is not zero when you fail to meet the most classes requirement, but it is significantly lower than if you had the most clauses because in this case, actually the probability is one.
Now this variable on which the threshold builds is called the assignment variable quite naturally. Sometimes it's also called the running variable or the forcing variable, but I would typically use the term assignment variable.
Perhaps sometimes forcing variable, but usually I think I would call it the assignment variable. Running variable, I do find a very good terminology.
If it is true that people just below and just above the threshold are really similar or perhaps equal in all characteristics, except for that they are above or below the threshold, then very clearly, our research designs are very similar to randomized experiments and therefore they can be analyzed, estimated and tested like randomized experiments.
We may have covariates there and these covariates may also help to reduce the variance of the outcomes. So like in randomized trials, it is often useful to include covariates in the regression analysis, which we have to do in our D designs, but we have to make sure that these covariates
have been realized prior to the assignment variable.
Then they are called baseline covariates.
Obviously, covariates must not be affected by the assignment variable, must not differ for people or must differ systematically for people who meet the threshold and people who do not meet the threshold.
So the distribution of the baseline covariates should be the same, just above and just below the cutoff point of the threshold.
Sometimes these covariates can be used to test the validity of the R and D design so we check whether actually the distribution of the covariates is the same.
There's a question.
For example, if we take the example of COVID restrictions, if one state introduces some kind of policy like 2G+, and another one didn't, will it still not be a sharp R and D?
It's not quite clear to me, and perhaps you can indicate this to me what you think is the outcome in this setting. Clearly, we have these restrictions and you may say 2G+, is a threshold, others not.
Oh, the number of cases is the outcome, number of COVID cases.
No, this is not a sharp R and D because even if you meet the 2G plus restrictions, there's still a possibility that you are impacted by COVID.
So the outcome can, for some individuals, still be COVID. So it is not the case that you are not treated in this kind of wording if you meet the 2G plus requirements.
There is some fuzziness, quite a bit of fuzziness, I would say, in this setting because it may be that you attract COVID despite of meeting the 2G plus requirements.
All right, let's move on. Here is a graphical illustration of the general idea. We have some type of assignment variable, let it be, say, unemployment duration or something like this.
And let the outcome variable be labor market efforts, for instance, and it may then be that you say, well, the longer you are unemployed, the higher is your labor market effort.
So this is the systematic relationship, but you are still on a relatively high level of benefits. But after six months, benefits are significantly reduced.
And this may have as an effect that you increase your effort of having a new job so that there is a discontinuity here, and the extent of this discontinuity would be the cause effect of not receiving the former benefits anymore.
So we could then look at people who are just above or just below the threshold, so just observations close to the threshold, that would be the typical setting in RMP designs, and compare their behavior before they reach the threshold and their behavior after they reach the threshold in order to isolate the causal effect.
And there's another question here, how can we test the validity of RD with the covariates? But what we can test is the hypothesis that RD gives us something like a randomized sample, right?
Because if we look at the covariates, and we have our sample, and we know which is the so-called treatment group, so people above the threshold and the control group, people below the threshold, then we can look at the covariates.
And if we find that the distribution of the covariates in the treatment group is the same as the distribution of covariates in the control group, then we can conclude that we have a randomized sample here and that RD is valid.
Yeah, thanks for the question, perhaps this clarifies things.
Any other questions?
I don't see any, so let us move to sharp RD.
We use sharp RD basically when the treatment status DI is deterministic and of course discontinuous function of a vector of covariates XI.
And we will here just focus on the simplest case where we just have one covariate XI.
Suppose that this is particularly simply structured, namely that XI is just a scalar, and when XI is greater or equal than some threshold X0, then treatment is given, then we have VI is equal to 1, and when XI is less than X0, then we have no treatment.
So we are precisely here, right? So this would be X0, then the threshold value where treatment is given or is not given.
When we have this type of sharp RD at a threshold X0, then there is no value of the covariate XI at which we would observe both the treatment and the control observations precisely because it is sharp RD, right?
In that sense, RD design is different from the settings which we have studied so far, in which it was always possible that we had overlap between treatment and control group with reference to the covariates.
So it was possible in the settings we have had so far that we had individuals in the treatment group and in the control groups which had exactly the same covariates, but that would not be possible in the sharp RD design.
For this reason, we can typically, we cannot afford to be as agnostic about the regression functional form in the RD world as we have been so far, because when we had this type of overlap, then we could very well compare people with the same covariates,
one being in the treatment group, the other one being in the control group, and then taking the difference between the outcomes of these people as the causal effect. So it was desirable to have this type of overlap.
But for us here in the RD design, we will have no overlap covariates. So technically, people are never exactly the same.
Even though we will typically work with just local samples, so samples which are local to the threshold variable, people are never exactly the same in the treatment group with the control group.
And due to this fact, we cannot compare people as well as we can in the settings we have studied so far. So we have to rely very strongly on the discontinuous function, and therefore we have to know the discontinuous function very well.
So we cannot be as agnostic about the functional form of the covariate assigning people either to treatment or not to treatment as we can be in the settings we have studied so far.
Let's look here at an example which is very similar to the one I have already talked to you, where I suppose that the true relationship is linear.
What I want to show you now is that it really depends on what kind of functional relationship between outcomes and covariates we have.
So here is the outcome variable, and here is the indicator variable, the assignment variable with the threshold value here at point five.
And then we have our observations which somehow scatter around the expected value of the outcomes given certain values of Xi. And it may be that the expectation of why I give Xi is just linear function of Xi as I have supposed here in this graph.
It may, of course, also be the case that the relationship is nonlinear. For instance, like I have it here where the observations scatter around some kind of sine wave.
Now, since we do not have overlap with respect to the covariate X, it may occur that a relationship which is actually nonlinear.
So where the true relationship is a nonlinear relationship is mistaken for linear relationship, because of course we never know what the true expected value of why I give Xi is.
So we never know exactly this straight line here or there versus the alternative that the conditional expectation of why I is perhaps a nonlinear function, which now is such as the dashed line here.
The only thing we have are the observations. We have these type of observations and with these observations, it is not completely impossible that we mistake a nonlinear relationship for linear relationship.
Now let us first study the case where the true relationship is linear and then later we come to the case where it is nonlinear and let's see what we can do about it.
So if we know that the true relationship is linear, then things are rather easy. We would then say, well, the expectation of why zero I give Xi is alpha plus beta Xi, right? So this is just this here.
A linear function and no causal effect because we look at why zero I, the expectation of why zero I given Xi is a linear function with some constant term and some stop parameter.
And suppose that the treatment effect tau is constant and the same for all individuals, then we would have y1i is equal to y0i plus tau. So the same set up as we have always studied in the potential outcomes framework.
These two equations here, two and three, then lead us to a regression where we would regress yi on first two variables which capture the conditional expectation of why zero I give Xi.
So we regress on a constant and we regress on Xi.
And then we also allow for the individuals having been treated. So we also regress on a treatment indicator di and the regression coefficient tau of this regressor here would be the estimate of the causal effect.
We know that di is a discontinuous function of Xi. So di and Xi are actually not independent regressors, but actually these regressors are correlated because di is a function of Xi.
It is not only correlated in the stochastic sense, but it is truly a deterministic function of the Xi.
But that is not the most important or most significant difference between the Xi and the di of Xi. But the important thing is actually that di is a discontinuous function of Xi.
Whether or not di of Xi is nonlinear or linear is not so important, is not the key difference. Let us see why this is the case.
Well, suppose that the true relationship between the expectation of y0i given Xi and Xi is nonlinear.
So in this case, we would have that this conditional expectation is some nonlinear function of Xi. However, it would be a continuous smooth nonlinear function f of Xi.
With this fact that we have a nonlinear relationship, change anything substantial? Well, not so much in truth, because the regression equation by just now constructing the analog of what we just did in the linear setup,
the regression equation would be yi is equal to the nonlinear function f of Xi plus tau di plus ui.
And this regression equation, which has a nonlinear term now, we can approximate, let us say, by a polynomial in Xi.
So, you know, a smooth nonlinear function can usually be approximated by the Taylor approximation as a polynomial of a certain degree.
So we could decide, let us say, for order p of the polynomial, and then we would approximate the nonlinear function here as alpha plus beta1 Xi plus beta2 Xi squared plus plus plus beta p Xi to the power of p plus then the treatment variable as tau di, here is the causal effect, plus some error term ui.
So here I had an error term ui, and here I have an error term ui, perhaps I should call it vi.
It's easier to understand the difference, or acoustically it's easier to distinguish between the two, so let's call this ui, and this here is vi, not the Greek letter nu, but rather Latin letter v.
And this vi is of course different from this ui because this polynomial here is only an approximation to the true linear function.
There may be some unexplained rest, right, and this unexplained rest would be in this error term here, and that would be the difference between the vi and the ui.
Now clearly this equation four with a polynomial is just a special case of, sorry, the equation four with a linear term is just a special case of equation seven with a polynomial, right, because four with a linear case is equal to seven when the order of the polynomial is just equal to one.
Now the di is again a deterministic function of Xi.
Therefore, the OLS estimation of equation seven yields a consistent estimate of the causal effect tau.
If it is true that f of Xi is indeed a polynomial of order less than or equal to p, right, so then we are off the hook when f of Xi is a polynomial of order less than or equal to p, because then we have really captured the true function in equations seven.
And in this case, in this case equation seven, which we have here, would give us the true relationship and the vi would actually be equal to the ui.
But it can of course be the case that this polynomial is not a perfect approximation to the nonlinear function.
What kind of problem do we run into then? Well, then the residual vi would be correlated with the Xi.
Actually, perhaps I'm now a little bit too fast. Let me just continue this sentence here where I interrupted myself. We get a consistent estimate of the causal effect tau if f of Xi is indeed a polynomial of order less than or equal to p.
So if the equation seven perfectly captures the nonlinear equation f, and if the Xi and the vi are uncorrelated, because obviously we must not have correlation with regressor and error term of the equation, the way we want to get a consistent estimates.
The covariance between Xi and vi must be equal to zero for all i's.
But this of course requires that the polynomial in seven is a sufficiently good approximation of the true nonlinear function. Strictly speaking, it requires that it is a perfect approximation to the true nonlinear function so that it does capture all of the nonlinear function.
If that is not a case, if this were not the case, then we would have some remaining nonlinearity in the conditional mean function, which we have not accounted for.
And this remaining nonlinearity may be mistaken as a jump due to treatment. So there's big danger here that we in this case would estimate the causal effect with some bias if there's remaining nonlinearity.
How can we reduce this possibility that we do not capture the nonlinear function well. Now, exactly by what I already have advocated from beginning onwards when it started speaking about regression discontinuity by looking just at small neighborhood of the threshold value x zero.
So when you go back to, let's say, this function here, and suppose the true function is the nonlinear function.
Then we can of course the dashed function which you see here.
Then we can of course try to approximate this nonlinear functions here by some polynomial.
And if the order of the polynomial is high enough, we will capture this sufficiently well.
But what we can also do if we have sufficiently many observations is that we say we just look at a small neighborhood around the threshold value of 0.5.
So for instance, just the values between 0.4 and 0.6.
And if we now look at the dashed function between 0.4 and 0.6, we see that is easy to approximate. This we can already approximate very well by a linear function, apparently.
We don't need a high order of the polynomial, which is supposed to be the approximation. But even order one would already do with p equal to one, we would have a linear function.
And that seems to capture the behavior very well if we just confine our attention to the neighborhood of the threshold value.
So for instance, on the interval between 0.4 and 0.6. And this is what is typically done in regression discontinuity analysis.
Okay, so if we focus on a small neighborhood of the threshold value, then the exact shape of the nonlinear function is not so important anymore.
But of course, we can do this only if we have sufficiently many observations in this neighborhood.
RD designs which make use of specific functional forms are typically called parametric. So if we use some type of polynomial approximations or other type of approximations, then we call the RD designs parametric.
If the RD designs, however, are confined to a small neighborhood of x naught, then perhaps we do not need a specific functional form. And therefore, these type of RD designs are often called non-parametric.
I'm afraid. Yeah, I think I will start here with this terminology, the distinction between parametric and non-parametric because I won't quite be able to complete the sharp RD design in today's lecture.
We do this then tomorrow. And then we move on to fuzzy RD designs, which are a little bit more complicated.
So I would conclude the lecture now unless you have further questions. Also, if you like, questions relating to the current exam.
If you do have any questions, then please raise your hand and I will wait before I conclude the lecture until you have typed in your question in the chat.
But if there is no indication and I don't see any, then I will say bye bye. Have a good afternoon. And oh well, there is a question now.
So I will wait a minute, please type in your question.
I didn't pass the message actually.
Sorry, but I would like to continue COVID example in case the treatment is the number of cases and what?
Well, the treatment was not the number of cases, right?
In case the treatment is the number of cases and why is the introduction of 2G plus or not?
Oh, so you reverse it now. You now say that the outcome is the rule which is being set.
What can we have in this case, sharp or fuzzy? Then we wouldn't have, in this case, we wouldn't have RDD at all, I would say, because then there is no threshold which has been set as a covariate.
In this case, what you, in your question before, labeled to be the threshold, namely the 2G plus restrictions, is now the outcome.
So then it is not a RDD design at all.
I hope this answers your question.
Okay, any other question or if this question has not been satisfactorily answered, then please raise your hand again.
Yes, please.
Suppose it's mandatory to introduce 2G plus above a certain number, I see.
So the rule is you are required to impose 2G plus.
Yes, that's possible. Then the threshold basically is again the 2G plus, but then again the outcome is not 2G plus, but the outcome would again be the number of cases, correct?
And then it is the same answer that I gave on your first question. The number of cases is a fuzzy response to the threshold and then to the rule which has been established.
Does this answer your question? If not, then please raise your hand again.
There's another.
Okay, that's just a thank you and you'll think about it. Yeah, that's good. So see you then tomorrow and have a good afternoon.