What you see on this slide again is the causal model we are using from now on, and which
I have introduced in the last lecture, which I summarize again just for your convenience
in the next three equations, so equations one to three of this set of slides.
You remember that I introduced a notation which distinguishes between different outcomes
denoted to Y1i and Y0i, which indicate that individual i either did receive a treatment,
in which case the outcome would be Y1i or did not receive a treatment.
It is not quite true. What I just said, it is not the fact that they did receive a treatment,
but rather these are the potential outcomes. In case that individual i received a treatment,
the outcome would be Y1i, and in the case that individual i, so the same individual did not
receive a treatment, would not receive a treatment, then the outcome would be Y0i.
These are just potential magnitudes, not both of them are observable. It depends on the treatment
status, which of the two outcomes we observe, and the other one is just a theoretical construct,
which we have no knowledge about. But in terms of theoretical constructs, we can use them and
explain the observations we actually have at hand, which would be Y1i, so this is the observed
outcome of individual i, and by definition this is equal to the outcome in the case of non-treatment
plus the difference between the outcomes in the case of treatment and in the case of non-treatment
multiplied by a dummy variable Di, which indicates whether individual i has received a treatment or
has not received treatment. So having received a treatment would mean Di is equal to one,
and having not received the treatment would mean that Di is equal to zero. So now it's easy to see,
depending on the value of Di, that the observed i outcome is either Y0i or Y1i, namely exactly in
those cases in which Di is equal to zero or Di is equal to one. And then we can define the causal
effect of some treatment as delta i, which is just the difference between the two potential
outcomes, because this difference here means everything else is the same, just the treatment
is different, so delta i, the difference between the two outcomes, is the treatment effect.
And therefore equation one can also be written as Y i is equal to the outcome in the case of
no treatment plus the causal effect times the dummy variable Di, which indicates whether
individual i has received treatment or has not received treatment. Now we were looking at this
set up in the last lecture and considered an estimator of the causal effect delta had naive,
which would just consist in taking the mean outcome of all the people that have been treated
minus taking the mean outcome of all the people which have not been treated. That we could think
of as the average treatment effect, or it is an estimator of the average treatment effect.
We stopped last lecture actually with this somewhat messy looking set of computations
and equations here, which showed us that the expected value of this estimator delta had naive
is equal to the sum of three terms, one of which is the term we actually aim at, which would be
the delta which measures the average treatment effect. And then there are two biases which come
on top of the desired result of our estimator, one is a differential treatment effect bias and the
other one is a selection bias. And I explained to you in principle what these two biases are,
namely differential treatment effect bias is the expected difference in the treatment effect
between the treatment and the control group. In this case, multiplied by the fixed proportion
of untreated population, but that's of lesser importance. The importance is that the important
thing is that there is a difference in the treatment effect between the treatment and
the control group. I will give you an example now in the next slide to make this perhaps somewhat
more concrete so that you better understand what exactly is the differential treatment effect bias
and what is the selection bias that you can distinguish the two. So the selection bias is
the difference in the average outcome in the absence of treatment between treated and control
group. So this was where we left off last lecture. So now let's look at an example where you may
imagine that we study the causal effect of college attendance on some measure of cognitive ability.
So the treatment we apply to our group of individuals is college education. You may think
of some population of young people and some receive college education and others do not
receive college education. So the treatment is college education and the effect is some measure
of their intellectual capacity after college. So we call it the cognitive ability. Now the empirical
finding we have is typically that we find higher cognitive scores, so higher cognitive ability
for those people who have attended college. Which probably is not very surprising.
Let's look at the potential reasons for this finding. So I give you three reasons for that.
The first reason is actually the reason that we would like to have that education increases
cognitive ability. So this would mean that there is an average treatment effect. Some causal effect
actually we can still debate the question whether it is the average treatment effect or perhaps the
average treatment effect on the treated. But in the way in which we set up the expectation of
delta head naive, we decompose the expectation of delta head naive into the three components here.
This is why I refer here to the delta ATE. So the causal effect measured as the average treatment
effect over the whole population. And it may just be the case that education increases cognitive
ability. That's quite plausible. However, there may also be a selection bias. College attendance
may on average be smarter even before college. So it may be the case that only the smarter ones of
our population actually apply for college or are actually admitted to college. So already before
they receive college education, they may be on average smarter. Which means that in the two
groups which we consider, one of which has received the treatment and the other one has
not received the treatment, there was already a difference in their cognitive ability prior to
college education. Which we describe formally as the expectation of cognitive ability without
treatment. So y0i, the expectation of y0i for those who did receive treatment is greater than
the expectation of y0i. So the expectation of the cognitive ability without college education
for those which did not receive treatment. So which did not go to college. So that would be
the selection bias. Plainly speaking, the smart folks go to college and the number ones do not.
And then there is c as the third reason, the differential treatment effect bias. The differential
treatment effect bias in this example means that the college effect is stronger for observed
college attendance than it would have been for those who did not attend college if they had
attended college. So it basically means that for the college attendance, for those people who
are supposed to go to college, you will expect that their cognitive ability is increased by a
greater amount than for those people who would not go to college. Perhaps because they have less
intellectual ability. Therefore, we would formally write the expectation of the causal effect
is greater for the people in the treatment group than the expectation of the causal effect for the
people in the group of non-traded of those people who do not attend college. So the potential to
improve cognitive ability is greater for the treatment group than for the non-traded group.
And this is then the second type of bias, the differential treatment effect bias.
Now, here comes an exercise for you to familiarize yourself with this type of concept.
I give you here certain hypothetical outcomes in a little table. Outcomes which you have in your
disposal in this exercise, but which you typically would not have at your disposal or not all of that
you would have at your disposal in a real world setting. I'll come to this on the next slide.
But first consider the data that I give you here. Suppose that 30% of the population obtains a
college degree. So we set this percentage equal or we note pi, pi would be equal to 0.3.
And now we have two groups of people, two groups of people. The treatment group, which is denoted
by DI is equal to one. So those individuals, I, whichever one as the value of DI are the ones who
go to college. And then there's the control group, which is characterized by DI is equal to zero.
And now suppose the word is structured as follows. The mean of the potential outcomes, why one I
would be 10 for the treatment group. So I would say the mean, the conditional mean,
meaning the mean conditional on belonging to a certain group, either the treatment group or the
control group. So for the treatment group, DI is equal to one. And then we know if an individual
I belongs to the treatment group, so given DI, then the mean of the potential outcomes
in the case of college education is 10. However, so 10 measures the cognitive ability. However,
if the same group of people did not receive treatment, if they had not received treatment,
then the potential outcome or the mean of the potential outcomes, I should say for the same
group of people, which actually did receive treatment in the case they did not receive
treatment is just six. So that's the causal effect. 10 minus six is the causal effect on
the group of people who receive treatment. And here we would have the number, the same numbers
for the control group. So eight here would measure the average of the potential outcomes
in the case that they do receive college for those people who actually do not receive college
education. And five measures the average of the potential outcomes, why zero I, for those people
who do not receive treatment in the case that they do not receive treatment.
So this is sort of the complete picture of the world as it is. And in this exercise,
you know those numbers in reality, you would not know those numbers, not all of those.
The exercise simply consists in compute delta hat naive, compute delta hat ATT. So an estimator
for the average treatment effect on the treated an estimator for the average treated effect on the
non-treated and an estimator for the average treatment effect. So compute these four estimators
or suggest estimators on how you can, how we can estimate them from these data here and also
compute the selection bias and the differential treatment effect bias. And of course the total
expected bias with these numbers. Now, as I already told you in a real world setting,
you would not know all the numbers in the table I just gave you. Unobserved would be the numbers
in red here. Unobserved would be the result of cognitive ability in the case of college education.
So index one here for the group of people who actually did not receive college education. So
we just don't know in a real world setting this number here. And similarly for the treatment
groups or for the people who did receive college education, we do not know what the
average potential outcome of these people would be if they had not received college education.
So we would also not know the six here. When you solve the exercise on the previous page,
so when you compute these estimates here or suggest estimators for them, then you will see
that you actually need for some of the estimators, some of the red numbers here. So the econometric
problem we will talk about in the next lessons is actually find good estimates for the means of the
unobserved potential outcomes. So find good estimates for the red numbers here. This is a
question we will address in the next lectures. We haven't addressed it at all. So for the example
just tries to show you what we are actually doing. I mean what kind of information we have
and what we need to have in order to answer the cause of questions that we may want to answer.
Note please that delta hat naive uses just the known numbers in black. I see there's a question
I'll take it in a minute. So delta hat naive the estimator we have already discussed uses
just the known numbers in black. And delta hat ATT requires one of the red numbers because it
requires the mean of the potential outcomes in the case of not having taken college education
for the treated. Quite clearly because this is the average treatment effect on the treated. So we
would need to know how the treated would have fared had they not received college education.
So we need this number here which is actually the red six. And similarly the average treatment
effect on the non-treated estimating this average treatment effect as delta hat ATT
requires the knowledge of the average treatment of the potential outcome in the case of college
education for those people which did not receive college education. So we would need to know the red
number five here. And finally the causal effect ATE the average treatment effect requires the
knowledge of both of the red numbers from which you should infer that it is informationally and
therefore econometrically less demanding to estimate the ATT than to estimate the ATE.
And similarly it is information less demanding to estimate the delta hat ATNT so the average
treatment effect on the non-treated than estimating delta hat ATE. As I showed you last week delta
hat ATE is actually a weighted mean between delta hat ATT and delta hat ATNT so between these two
estimators here and therefore very clearly for delta hat ATE you need knowledge of both this
number and that number. Now oh well yeah there was somebody raising his hand let me have a look at
the chat here. Yes the question is whether the potential outcomes in red are the counterfactual
that's exactly it. Yes the potential outcomes numbers in red in my table are the counterfactuals.
I think this question comes from somebody else as compared to the one who here raised his or her
hand. Please I don't know how to pronounce the name Zürier. Would you also type in your question?
Or was this the question?
I don't see any question coming in so perhaps this was it. Now here it is all right okay
somebody accidentally clicked the button all right okay um good. Now the ATT is computationally
less demanding than the ATE as I have just said and in fact it is often the case that we are mainly
interested in the ATT. I gave already the example of hospitals and hospitalizations.
We are not really interested in the effect hospitalization has on the average population
because we know many people in the population don't need hospitals because they are just healthy
right so the average treatment effect on the population is not really interesting because
from many people the individual effect would just be zero so averaging over a great number
of people which have a treatment effect of zero because they are healthy in the first place
would mean that the treatment effect on the sick people would be of rather limited importance in
the average and therefore perhaps the average treatment effect is very small even though for
some people which are really ill the treatment effect is large since they need
hospital care but what we are actually interested in is the average treatment effect on the treated
and not the ATE. The same holds for many other questions I give here the example of special needs
schools so schools for instance for the handicap for people who are blind or people who are deaf
or mentally handicapped or whatever they need special needs schools and what we are interested
in is of course the question how is the causal effect of these special needs schools for those
in need of those schools so for the handicapped we are not interested in how big is the causal
effect of special needs schools for people who actually don't need a special needs school so
it is the ATT which in many applications is actually the interesting causal effect.
Now what I showed you last lesson was that delta het naive is not a good estimator for the ATE
so for delta ATE it is not a good estimator because we have the differential treatment
effect bias and the spectrum bias but you may not ask the question why actually should delta het naive
be an estimator for delta ATE in the first place you may also say that perhaps delta het naive is
by construction closer to an estimator for the ATT so perhaps you may argue that delta het naive
is the average outcome for the treated minus the average outcome for the untreated so you could
ask the question well isn't that perhaps an estimator for the average treatment effect
for the treated we have shown it's not a good estimator it's not an unbiased estimator for the
delta ATE but is delta het naive possibly an average sensible estimator an unbiased estimator
perhaps for the ATT well if you listen carefully what I just said then you will probably already
come up with the right answer and say probably not probably not because you go back we know delta
het naive makes use just of the numbers in black okay and in order to estimate delta het ATT we
also require one red number one counterfactual so it is not really likely that delta het naive is a
good estimator of delta het AT but perhaps it's a better estimator of delta het ATT than it is an
estimator of delta het ATE because here it's just one counterfactual which we need whereas for delta
delta het ATE we would have needed two estimators two two counterfactuals so let's look how delta
het naive relates to an estimate of delta ATT the average treatment effect on the treatment
and this we can do in a way completely analogous to the computation we have done last week actually
it is somewhat easier for a reason which you will understand in a minute if we look at the expectation
of delta het naive we can start in exactly the same way we started last way last last lesson by
saying where we take the estimate the expectation of the estimator and the estimator is the sample
mean of the treated group minus the sample mean of the non-treated group so it's the expectation
of the potential outcome in the case of treatment for the treatment group minus the expectation of
the potential outcome for non-treatment given that individuals are in the group of non-traded
okay so then we proceed in the same way as i have already done last time we add the number we are
actually interested in namely the delta ETT that's the thing we want to center our attention on and
what we do is we just add it once here and then we subtract it immediately which we do here
as you see by this large bracket here this is equal to minus delta ATT so it cancels against
the plus delta ATT here these first two terms are exactly the same terms as these terms here
and then we subtract here delta ATT where what is delta ATT that's the treatment effect the
average treatment effect on the group of treated so we would look just at the group of treated
delta i is equal to one here and delta i is equal to one here and we take the expectation of the
potential outcome in the case of treatment minus the expectation of the potential outcome in the
case of non-treatment just for the treatment group so that's delta ATT well here i put a
negative in front of that so i subtract it in order to make up for adding delta ATT here
and then we just need to compare the blue terms here and the green terms here and you see that
the blue terms cancel right so that's the same thing plus here negative here so this cancels
and then we have negative e of y0 i for the non-treated minus well actually double minus minus
times minus plus plus expectation of the potential outcome of non-treatment for the treated so what
we are left with is this difference here expectation of none of the potential outcome
in the case of non-treatment for the treated minus expectation of potential outcome in the
case of non-treatment for the non-treated so that's the selection bias so we see that the expectation
of the naive estimator is equal to the delta ATT we actually aim for plus the selection bias
which is some progress because in terms of the ATT we had the parameter we desired
plus selection bias plus differential treatment effects bias but the differential treatment
effects bias has now vanished so we are confronted with just one bias which tells us that if we find
a way to eliminate this bias here then actually we can just use the numbers we have at hand
namely the black numbers in my table to compute delta hat naive and we would know that its
expectation is the parameter we would like to estimate namely delta ATT so the problem now
amounts to eliminating the selection bias and we'll do this actually in the next set of slides
but currently we don't know how to do it perhaps you have a vague idea or even better idea but
I'll tell you in the next set of slides currently we still have the result that delta ATT naive
is a biased estimator and suffers from selection bias. Now please recall that in slides two I ask
you to pick a research question let me just go back to remind you of what I talk about because
it is some weeks ago that we did that but it was this thing here right I asked you to do a
thought experiment and in this thought experiment you are a researcher who has to research a certain
question and I gave you a long list of possible questions you may be interested in but you are
absolutely free to think of any other questions you may question you may be interested in but I
told you pick one of those research questions here or your own research question in the sense of a
concrete problem you would like to research to do research on you would like to investigate in
reality and during the course of the lecture I will come back to this research question
and then you should sort of continue your thought experiment by answering certain questions
I will give to you in which you basically as a thought experiment apply certain concepts
and certain methods which I teach you during the course of the lecture so you remember I hope and
recognize again those questions here I hope you know which question in your mind you picked as
your personal research question be now because now I return to this setting with the first such
exercise which comes here
for the research question you have chosen answer the following question so that that's an exercise
first what is the treatment in your particular research setting
second what are the potential outcomes in your particular setting
what is the treatment group and what is the control group well that's usually very
strongly connected to the question what is the treatment and what kind of selection bias would
you assume where this question actually asks you try to assess before knowing anything else about
data whether the selection bias is positive or negative with respect to the potential outcome
you're interested in
okay let me finish this set of slides with a few remarks on causal language which are actually
trivial but still it may be useful to quickly go through them I didn't I distinguish here between
causal and non-causal language where causal language always describes some type of causality
relationship so it describes cause and effect relationship which means that it describes
the hypothesis that one variable affects some other variable and often but not always causal
language also indicates the direction and possibly even the sign of the effect which the cause has on
the effect sorry I think I misspoke a little bit it obviously always indicates the direction namely
from the cause to the effect I mean that's the definition of cause and effect but not always
however often it also indicates the sign of the effect but sometimes it does not so the sign does
not necessarily belong to causal language but it may be a part of the causal language
the opposite case is non-causal language which just describes correlations and therefore describes
a certain statistical relationship without implying that one variable affects another
variable in some type of cause and effect relationship so to give you some example
causal language would be that education improves health because education improves
really suggests that education is the effect of better health right so that would be causal
language which establishes education as the cause and health as the effect the sentence
does not need to be true I mean we know that it's not necessarily true that education improves
health but it is causal in the sense that it formulates a causal hypothesis
whose truth we may still do our research on but we have the hypothesis which establishes some type
of hypothetical the same thing would be true for income increases longevity so the same type of
thing we would identify in this causal language income as the cause and longevity as the effect
or music lessons improves kids brain development is also a hypothesis may not be true at all
but it very clearly suggests a causal effect of music lessons on the development of a child's
brain by contrast non-causal language would for instance be educated people have better health
when we say educated people have better health then we just describe a statistical correlation
we describe that with the data we see that people with better health are better educated people who
are better educated have better health but this does not necessarily imply that there's a causal
relationship between education and health as we have expressed it in the first example here
education improves health or non-causal language would be rich women live longer
that again I mean you may think it suggests some type of causality and in fact perhaps it does
but taken literally the sentence rich women live longer just describes a correlation
and the same thing would be true for kids who take music lessons have bigger brains here we
measure two things and we see there is a correlation between two things namely having
music lessons children having music lessons and having bigger brains but whether actually
the bigger brains are caused by the music lessons that's a completely different thing so the
language here is non-caused but as I already said non-causal language often suggests causality and
sometimes it is abused there's a misuse of non-causal language to intentionally suggest causality
in particular the media very often confused both um intendedly or unintendedly that we may leave
open here's an example taken from media reporting the headline was TV raises blood pressure
in obese kids so overweight kids so TV raises blood pressure in obese kids suggests that
watching TV causes overweight kids to suffer from increased blood pressure so the headline
very clearly is a headline which uses causal language because TV is the cause for raising
the blood pressure in obese kids so that's the sensational headline which the media use
the article then says so the text of the reporting says research has found children who watch two
to four hours of TV for 2.5 times more likely to have high blood pressure compared with those who
watch less than two hours of television a day so what the article says is a mere correlation
the article says we observe a correlation between high consumption of TV and the likelihood of high
blood pressure that's a positive correlation there but what researchers have found is just
the correlation they have not found evidence or causality or at least the article does not report
that researchers have found evidence of causality it could actually be that the causal
relationships are different that say there is a third thing which has a causal effect on both
watching TV and having high blood pressure one very easy explanation could for instance be
reverse causality it could be that people that kids excuse me children who are overweight
suffer from high blood pressure so suffer from high blood pressure and this because they don't
feel so well they have high blood pressure they are overweight causes them to watch more TV than
people with than children with normal weight so what the article says may be completely true
but it's just correlation it is no proof of causality at all and the headline is therefore
completely inadequate because it makes causality out of something which is just correlation
i give you a link here where you can find more such examples where correlation and causation
are confused in media reporting
we do have the distinction between correlation and causality also in language but it is not
really precise so in general we may say that vocabulary like increases or reduces
prompts makes improves boosts that this is vocabulary which suggests causality
also vocabulary like consequences impact cause reason obviously effect or grammatical structures
like because sins due to all of this is causal language whereas non-causal language which
use wording like is associated with is higher or lower than so you make certain types of
comparisons that you speak about associations correlations relationships that you use verbs
like get have linked indicate relationship but more or less use words like tied connected related
tend to or something like this when you write a scientific paper you should be
very careful about your wording because obviously your readers will interpret language in the way
in which words suggest it should be interpreted so they will infer either causality or correlation
from what you describe so usually you should be careful in choosing your words and use
words which suggest causality only if you can establish causality and otherwise use language
which is non-causal and just describes facts which are observable in the data
all right for your research question once again write a potential answer to your research
question and write it as a causal statement and then by contrast think about how your statement
would look like in non-causal language okay for time reasons we will skip the last section of
this set of slides so let me ask if there are any questions um i'm sure if this is the old one
no this was the old one uh any other questions concerning this set of slides let's for
please raise your hand if you have any
this is not the case then i will leave the set of slides and go to slides five
slides five is on randomized trials and the readings which i suggest here are the same as
for slides four so so again the two books by angrist and bishke now it's chapter one and two
in the 2009 book by angrist and bishke and is chapter one in mastering metrics the 2015 volume
okay what we speak about now is one way to solve the selection the selection problem
recall that delta had naive the estimator we had just analyzed did in its expectation differ from
the att just by the selection bias so as i told you if we can get rid of the selection bias then
actually we know how to compute the causal effect because we can just use delta hat naive
so here i teach you one way to solve the selection problem the section bias to do away with the
selection bias is solving the spectrum problem which is random assignment of the treatment
what does random assignment of the treatment mean it means that the researcher randomly assigns
individuals to treatment and control group so the researcher himself is also the guy who conducts
the experiment which is often not the case often experiments are conducted by other people than
those which do the statistical evaluation later on perhaps for completely different purposes or we
have what we call natural experiments so nature makes some type of experiment exposes some people
to some influences and researchers come in later and analyze the data but obviously they have not
been in the position to control the experiment in the sense that they were able to decide which
individuals receive treatment and which individuals do not so random assignment means that the
researcher himself is in the position to randomly assign the treatment to the different individuals
so that there is no systematic effect behind the decision who receives the treatment and who does
not receive the treatment so for instance in our hospital example we clearly do not have
random assignment because only the sick people receive the treatment and the healthy people do
not receive the treatment so there is a systematic reason why some people receive treatment and
others do not but if we drew a lottery on all people in the population assigning some of them
to receive hospital treatment and others not to receive hospital treatment then we would not have
selection bias anymore it would not be very sensible to do that because then some of the sick
people would be assigned not to receive hospital treatment and many healthy people would probably
be assigned to receive hospital treatment so I mean in terms of public health this would be a
suboptimal outcome but in terms of statistics it's an ideal outcome because we would solve the
selection bias right we could easily estimate the ATG all right we have random assignment of
individuals to treatment and control group and that means that with respect to the distribution
of all characteristics in the population the treatment and the control group are just the same
with one exception and that's the treatment status so for all possible characteristics of
the individuals in our sample the expected value of those characteristics is the same
in the treatment group and in the control group and actually not only with respect to the expectation
which is just the first moment but with respect to the whole distribution of characteristics
the distributions are exactly the same in the treatment group as in the control group
so there's no systematic difference between the treatment group and the control group
with the one exception that the treatment group is being treated and the non-treatment
group the control group is not being treated which is the necessarily the necessary distinction
between the two groups and this allows us to compute the treatment in fact because we know
the two groups are identical statistically identical identically identical in distribution
with exception to exactly the treatment and so we know whatever the difference in outcomes is
can only be due to the treatment and therefore we have the treatment.
Note that not only for the observable variables we would need to ensure that the distribution
of those characteristics is the same in the treatment group as in the control group
but it must also be true for unobservable characteristics which of course is a problem
because unobservable characteristics by definition we do not observe so we cannot
check whether the unobserved characteristics have the same distribution we can only hope for
this to be true because we have a random assignment we just draw a lottery it's just
a random experiment which we do with the population we randomly draw people from the
population then we would say well the expectation is that also unobservable characteristics must be
distributed in exactly the same way if the sample is large enough in the treatment group
and in the control group because it's random assignment so random assignment is statistically
actually the perfect way to solve the selected problem. Now if we have this random assignment of
the treatment then we can think about the effect the potential outcomes have on the treatment
variable PI. The result is actually that potential outcomes do not have any or it's
not so much a result it is actually already included in the premises of the random assignment
potential outcomes also do not have any effect on treatment status so that's actually quite
important if there were a correlation between potential outcomes and treatment status then
this would mean that we still have selection problem because potential outcomes are different
in the treatment group from as compared to people in the control group. Random assignment implies
that the DI the treatment variable is independent of the potential outcomes and therefore we can
write the expectations of the observed variables as follows the expectation of the outcome variable
of the observed outcome variable YI in the treatment group minus the expectation of the
observed outcome in the control group is obviously equal to the expectation of the potential outcome
in the case of treatment for those in the treatment group, minus the expectation of the
potential outcome in the case of non-treatment Y0i for the non-treated because we know for the
non-treated we observe the potential outcome in the case of non-treatment and for the treated
we observe as yi the potential outcome in the case of treatment so we observe y1i.
1i. So this explains this equality here. And then we can also infer that this is equal to, again,
the expectation of Y1i given Di is equal to 1, so potential outcome in the case of treatment
for the treatment group, minus the expectation of the potential outcome in the case of non-treatment
for the treatment group. And this here is one of the red numbers, right? That's a counterfactual.
We do not know what the potential outcome in the case of non-treatment for the treatment group is
because the treatment group has been treated. So we do not observe the outcome in the case of
non-treatment. So why can we in this equality here replace this term which gave us the potential
outcome in the case of non-treatment for the non-treated by the expectation of the potential
outcome in the case of non-treatment for the treated? Well, we can replace this because we
have random assignment because we know that the potential outcomes are actually independent of
being part of the control group or being part of the treatment group. Potential outcomes are the
same since we have random assignment, right? This is why we can replace Di equal to 0 whereby Di is
equal to 1 since the potential outcomes in the case of non-treatment must be the same on average
and expected value in this case, expected value, must be the same in the treatment group and in the
control group since we have randomly assigned individuals to treatment and control group, right?
Yeah, and so here we have the ATT. This is the average treatment effect on the treated because
now we refer just to the treated Di is equal to 1 here, Di is equal to 1 here. So we just refer to
the treated and we look at the expected value of the potential outcome in the case of treatment
for each individual minus the expected outcome in the case of non-treatment for each individual in
the group of treated. So that's the ATT here which makes use of one of the red numbers and in this
case we can estimate this red number here, this thing here, right? By a black number by this thing
here because we know that we have random assignment so we know that the red number should be equal to
the black number. So that's one way to solve the selection problem. Here again it is the independence
of the potential outcome y0i and the treatment which allows us to identify this term here,
expectation of y0i for the non-treated with expectation of y0i for the treated. So we have
the independence of y0i and Di and let me also state that we not only have the independence
between y0i and Di we also have independence of y1i and Di so both potential outcomes are
independent of the treatment. Now we may actually simplify this further and thereby fully exploit
the property of random assignment by writing that this average treatment effect which we had in the
last equality on the previous slide. So expectation of potential outcome in the case
of treatment for the treated minus expectation of potential outcome in the case of non-treatment
in the treatment group. Well that's the expectation of the difference between the two potential
outcomes obviously in the case for the group of the treated and since the potential outcomes
are independent of the conditioning variable treatment or non-treatment here that's also
the same thing as the unconditional expectation of y1i minus y0i. Good, let's look at an example
here. Suppose we do what I already described before as a nonsensical public health experiment
but ideally in statistical terms we assign hospital treatment randomly to everybody in
the population. Well what would happen in this case if hospitalization is randomly assigned in
the in the population then the effect of hospitalization on the hospitalized is the same
as the effect of hospitalization on a randomly chosen patient. So the effect of hospitalization
on those which have been treated on those which are hospitalized is the same then as the effect
of hospitalization on any randomly chosen patient. So we know then that the ATE is equal to the ATT.
That's a general property of random randomization and there is no difference between ATE and ATT
in randomized control trials and randomized control trials are often abbreviated RCTs.
Well and again this of course hinges on the fact that the random assignment of DI
just eliminates the selection bias. Here's a historical example of a randomized trial versus
a non-experimental survey. And this historical example, so something which really happened in
the United States was that physicians and other people suggested that women which are in their
menopause that they should be subjected to some kind of hormone replacement therapy because you
know middle-aged women when they reach menopause suffer from various symptoms which are at least
uncomfortable due to a change in their hormones and therefore physicians suggested that perhaps
they could be applied some type of hormone replacement therapy abbreviated HRT for middle-aged
women to reduce these kind of symptoms. Well and then there was actually a health
study which was called the nurses health study because this study used medical checkups of nurses
women working professionally as nurses and this study showed that HRT participants so nurses
in their middle ages which had received hormone replacement therapy showed better health than
those which did not. And that gave some boost to hormone replacement therapy this nurses health
study or this evaluation of HRT on the basis of data from the nurses health study. The problem
however is that nurses are not a random sample of middle-aged females. So later scientists came up
and said well we don't really know where the nurses are representative of all women in the
population. Perhaps nurses have special features perhaps nurses are more aware of health issues
than other middle-aged women are and therefore nurses are not in all characteristics equal to
other women in the population and therefore we may actually have a selection bias. It is not so
clear from the outset why there should be a selection bias and what kind of effect it would
have but in principle since not everybody in the population is a nurse just picking the nurses for
this type of study raises the possibility of a selection bias. So researchers went ahead and
devised other studies for HRT in designing randomized controlled trials.
So they picked women randomly from the population and women in the respective age group and some of
them were given HRT and others probably just received a placebo so the women didn't know
whether they had additional hormones or whether they did not receive them so they were treated
all the same way and since the women were randomly picked from the population researchers knew there
cannot be any selection bias anymore. The interesting result was that this study which
used randomized controlled trial showed very few benefits of HRT and revealed serious side effects
that had not been apparent in the nurse's health study. So I write here the non-experimental data
because this was not a controlled experiment in nurses health study it was some kind of
arbitrariness which the researcher did not have under control to with just the nurses
but in a fully controlled experiment where assignment was random it turned out that actually
there were serious side effects which had not been detected by just looking at the data of nurses
and therefore we just see that non-experimental studies can be quite misleading since they neglect
the selection bias and it may not always be obvious that there is selection bias. So many people may
have read the nurses health study and said well who cares what whatever profession these these
women have they are middle-aged women like every other middle-aged woman and therefore we conclude
that the results of the study are representative for the population but it seems that this is not
the case that nurses are in some way a special group and that there is selection bias and in
this case the selection bias favored HRT much more than the randomized controlled trial did.
Another example would for instance deal with government subsidized
training programs on the top market. So you may think of certain government subsidies for people
workers which are disadvantaged for instance long-term unemployed people or drug addicts or people
with a criminal history you know they have higher hurdles to enter the job market more difficulties
to find a job even a rather badly paid job can be sometimes quite difficult so in any modern
country they are typically government subsidized training programs and these are expensive and
obviously you need to evaluate those training programs to see whether they create benefits
and perhaps also to see whether the benefits are worth the the effort in terms of costs.
So in this case there was a program that had some type of classroom instructions and some
type of on the job training for groups of disadvantaged workers to facilitate integration
in the job market in the labor market. The idea was of course to increase employment and earnings
for those people. Now initially there were studies based on non-experimental that is to say
non-random assignments comparisons of program participants and non-participants showing actually
lower earnings for the participants after training than a plausible comparison group had
or at least the studies argued that there was a plausible comparison group and by comparing the
participants of the government subsidized training programs and their labor market success after
training with the labor market success of the plausible comparison groups the study suggested
counterintuitively that participants who received the government training classroom and the job
training ended up with lower earnings than those people who did not receive the treatment but were
otherwise comparable so that would be quite puzzling. The point is here that this was still
a non-random assignment. I mean people in these studies argued that there is a comparison group
which is plausible so they argued this group should actually have approximately the same
potential outcomes but there was no random assignment there was not a controlled random
experiment which assigned participants of the program or yeah which assigned people to
participating in the program or assigned them not to do so there is still a concern that perhaps
there is a selection bias therefore other people conducted randomized evaluations of such
training programs and they generated mostly positive effects actually even there the effects
were not completely clear it is not always clear that the government subsidized programs had the
positive effects people had hoped for but the effects which were found in the randomized
evaluations were stronger more likely to be positive greater in size than the non-experimental
evaluations of the problem. So it may be that the initial authors were wrong in saying that
the comparison groups were plausible comparison groups so perhaps they were not as plausible as
the initial researchers had thought they should be.
Now in the social sciences in particular in economics randomized trials are not as often
encountered as in say medicine but they are also becoming more prevalent I mean the statistical
advantages of randomized controlled trials are undisputed and so people who design
certain experiments also argue for randomized controlled trials and apply for funds which
share finance the studies in such a way that random assignment is possible often random
assignments is more expensive than non-assignments than non-experimental designs. So there is a
tendency towards having more randomized controlled trials in the social sciences than we previously
had in medicine that is already the standard in many respect for instance if you think about
vaccinations now with COVID-19 of course the pharmaceutical companies which have developed
some type of vaccine now use randomized controlled trials to test the effects of the
vaccine on the population so they must be sure that in their treatment group those people which
receive the vaccine and in the control group those people which do not there is no systematic
difference in composition between the two groups except for the fact that one group receives the
vaccine and the other one does not. Let me discuss one last example which is sort of on the
borderline between economics and education so education research which often studies the question
which input actually fosters education or cognitive ability so what is most effective
in increasing the potential of school education for instance and one of the most expensive inputs
is class size because of course class size basically means the number of teachers the
smaller the classes are the more teachers must be higher for a given number of students so
teachers are expensive and therefore class sizes are some input in education
costly input and you may ask the question well how valuable is it actually how effective is it
what kind of results do we achieve by using many teachers in small classes
there was also a well-known randomized study in this field addressing exactly this question
which is called the Tennessee Star Experiment because Tennessee and it was designed to estimate
the effects of smaller classes in child education in kindergarten and primary school so the research
question is actually does the expense of smaller classes pay off in terms of higher student
achievement and this was what the Star Experiment was designed to answer the reason why it was
conducted here as a randomized controlled trial is that many studies of education prior to the Star
Experiment used non-experimental data and these non-experimental data suggested that there is
little or no link between class size and student learning so many studies actually came up with
the conclusion that students in larger classes had approximately the same cognitive ability
and then students in smaller classes or even better cognitive ability so that people were
arguing well there is no need to hire more teacher there is no beneficial effect of smaller classes
which is perhaps not the most intuitive result so other people questioned these type of results
and said that it is dangerous to take an observed relationship between class size and student
achievement at face value because taking just what we observe in reality means that we have no random
assignments of students to either big classes or small classes and there may be systematic
differences between students which attend the larger classes and students which attend smaller
classes the systematic difference may actually be that teachers decide to put the weaker students
in the smaller classes because they think they need more care right and the stronger students
are assigned by teachers to the larger classes which may then naturally have the effect that
the cognitive ability measured after some years of attendance of large class is greater than the
cognitive ability measured after students have attended the small size classes for a number of
years precisely for the reason that the weaker students the less able students the students with
problems at home or just the dumb students were assigned to the small classes on the reasonable
assumption that these students need more care so the non-experimental data can be greatly misleading
and there may be a severe selection bias because students in the students assigned to the large
classes had a propensity to having a higher cognitive ability already on the date of assignment
to the large class now a randomized controlled trial would overcome this problem by ensuring
that we just compare things which are fair to compare so comparing apples to apples
um and this was what the star experiment did um so by randomization students were assigned to
different uh different size differently sized classes but otherwise students were completely
comparable and then the results of the star experiment pointed to a strong and actually
lasting payoff to smaller classes and i will show you the results in a minute the general idea of
the randomized experiment is that the characteristics of the individuals in our studies
are successfully balanced between the different treatment groups in the easiest case just
treatment or non-treatment but in the star experiment there were actually three
types of treatment because they were also considering medium-sized classes so we may
also have somewhat more differentiated setup of treatment groups than the one we have discussed
so far even though we will leave it out in our discussion the question is what kind of
characteristics are there for students which we can measure well in the data there was actually
not that much of information on the various characteristics which may characterize a child
and its ability or its preconditions for certain cognitive abilities but some such
variables that were measured were for instance whether the child was entitled for free lunch
which is an indicator of the parent's income so when the parent's income was low then children were
were entitled to receive a free lunch at school student race obviously in the united states of
america was an important issue whether black or white or hispanic or chinese or whatever
and student age of course was also a characteristic which was important to observe
now in regular tennessee classes in 1985-86 the average class size was about 22 students
and the study involved 11 600 children so quite a big number the experiment now assigned the
students to one of three treatments namely small classes regular classes with a part-time
teacher's aid and regular classes with a full-time teacher's aid so we will leave out this second
class here and just compare the small classes with the regular classes with the full-time teacher's
aid this table here gives you some information on the characteristics that were observed after
randomizing the assignment of students to either the small classes or the large or in this case
regular classes and do not underestimate the problems of such an experiment if it is done
in reality because you meet stiff opposition by parents probably if you tell parents that
your children will be randomly assigned either to small or to regular classes and immediately
parents pop up and say why does my child not get the benefit of the small class it will argue well
my child actually is suffering from some kind of disability and should have the special treatment
in the small class so why is it now assigned to the regular class or people come up and say well
my child is fairly normal but so is my neighbor's child and they are best friends and now one is
in the small class and the other in the regular class so they can't be best friends anymore and
why is there this stupid experiment that I don't care I don't want my child from suffering from
some kind of scientific desire to find things out which I'm personally perhaps I'm not really
interested in so it is not always easy to protect because researchers don't have the legal power
to just set the conditions for schooling and say like dependency but they need to have
yeah they need to convince parents that it is useful to do so and of course they need
to convince government authorities that they stand up against some of the perhaps more
egocentric resistance which comes from parents in view of the scientific benefit which may reside
from a randomized controlled setup so what we see here is the distribution of three variables
which have been measured and the first three lines of the table whether a child receives free lunch
so basically income of the parents whether the child is white or Asian that was one group or
whether it is black or Hispanic that was apparently the other group and what the age of the child was
in 1985 and this is actually a table for a star in kindergarten so it's still before primary school
but as I said it doesn't really matter much I just want to show you that due to randomization the
distribution across these three characteristics here is very similar so about 47 percent of the
students received free lunch in the small classes and 48 percent received free lunch in
regular classes the t-test that these two means here right these are just the means of the
students in the classes so given di the t-test that these two values are the same as a p-value
of nine percent only you would perhaps think that is greater but given the great number of students
so eleven thousand six hundred observations are how many it was the standard errors were already
very small and even for this seemingly small difference between 47 and 48 percent I mean
the hypothesis of equality could not be rejected there was still a probability of nine percent but
that's not so much right not so much okay the composition according to race whether white
Asian on the one hand side or black Hispanic on the other side was 68 percent white Asians in
small size class and 67 percent in the regular classes again the t-test indicated
a probability of the null hypothesis of equality of 26 percent which is fine
and then age in 1985 was somewhat older than five years 5.44 and here also 5.43 so very very similar
actually the p-value was 32 percent that actually the age is exactly the same in the small regular
classes so at least according to three characteristics which we can observe randomization
apparently really solved the selection problem and therefore we can hope that for other characteristics
which we do not observe the selection problem was also solved by randomization now the attrition
rate is something I will not speak about much that's actually one of the problems because it
turned out that some people discontinued some parents discontinued the experiment
so they took their children out of the classes for whatever reason
maybe parents moved away took up a different job wanted to go to a different school or whatever
though so the attrition rate measures how many of the people of the students actually stayed
kindergarten over the course of the years and you see the attrition rates are different 49 percent
and 52 percent and the p-value that this is the same is actually low it's below five percent it's
just two percent so perhaps there are some hidden characteristics which despite randomization
are not the same in the sense that people had a different propensity to take the children out of
the classes they attended well and then line five here is class size in kindergarten obviously this
is a big difference of 15 children here 22.4 children here on average but that's just the
treatment this is what should be different we have small classes here and greater classes here
and then here is the outcome the percentile score of cognitive ability in kindergarten
which was 54.7 here and 48.9 here a substantial difference in outcomes which indicates that the
smaller classes had much greater effect on cognitive ability than the regular classes and
as you see this is highly significant yeah
okay so this is what we can learn from my discussion of the star experiment here
I think what I have written on this slide I have basically said yes I have said this
okay perhaps with exception of the last remark here the difference in means which we now observe
between treatment and control groups in the outcome variables can be obtained now from a
regression of the test scores on dummies for each treatment group by which I mean that we can now
turn to regression analysis of experiments and I will argue that this is actually a useful
tool for the study of causal questions because we can now actually frame the whole discussion
we've had in terms of potential outcomes as a regression equation by writing down a regression
such as this we take the observed outcome so for instance the cognitive ability measured in
the star experiment the observed outcome for each individual i and regress it on a constant
regressed on a dummy and and regressed on a dummy variable and then there's an error term
right so what we estimate I mean what we know is the di we do know whether the individual has
received treatment or has not received treatment the di is one or zero and the delta is a parameter
we want to estimate the alpha is a term parameter we want to estimate so that's just a regression
of the observed outcomes on a constant and on a dummy variable why does this help us in finding
the causal effect in randomized controlled trials we'll see with constant effects so assuming that
the causal effect is the same for each individual which of course is a strong assumption we may write
our measurement equation for the observed variable yi is equal to y0 i plus y1 i y0 i di
in exactly this form here because the difference between the potential outcomes here is just the
delta which in this case assuming that the delta does not depend on i is a constant parameter
for each observation i in particular here it is the constant parameter by which
the dummy variable which indicates the treatment is b multiplied di the alpha i here we will
interpret as the unconditional expectation of the potential outcome in the case of non-treatment
and so this is this variable here and we just take the unconditional expectation here
the ui is then the difference between the potential outcome of individual i
in the case of non-treatment minus the unconditional expectation over all the videos
of this potential outcome so if you write it this way here then you see that
this alpha here this delta and this ui are exactly equal to the expressions given below
these symbols here and then we would get exactly this equation here
which i introduced to you at the beginning of this lecture and the random part of the regression
would be ui so the random part of the potential outcome in the case of non-treatment
and delta would be the causal effect um next lesson we will think about the question whether
there is correlation between the di and the ui because as we learned the review of basic
econometrics there would be a problem if the di correlates with the ui but that's reserved
for the next lecture because time is almost over let me ask you is are there any questions perhaps
first um regardless of whether we go to interactive mode or or not um are there any questions then
please raise your hand
i think what i see here still the old one yes
any questions i don't see anybody raising uh his or her hand then it seems there's no need to go
to interactive mode and all is playing to you which i hope is really true um still don't see
anybody uh raising the hand so then that's it for today i will stop the recording here we will not
have an interactive meeting as apparently there is no desire and we see each other again on thursday
after so long