Thank you.
Welcome to the lecture on estimation and inference in econometrics. This is the recorded
lecture which is scheduled to be given on January 3rd. We stopped in the last lecture discussing the
case of bad controls. Bad controls were loosely defined as control variables as covariates and
causal regression which are themselves affected by the treatment whose effect on the outcome
variable we want to estimate. We have discussed the problem of bad controls in the context of
an example I gave to you of a study which tries to isolate the effect of college education on earnings
or wages. So a study which tries to find out by how much wages are increased if people
have a college degree, have undergone college education. And we have assumed that in this
study we have observations on videos obviously on their earnings or wages and on the college
status. So whether they have a college degree or whether they do not have a college degree and we
measure the college degree by a dummy variable ci which is one if there is a college degree and ci
equal to zero if individual i does not have a college degree. But we also have information on
the occupational status of the individual. So in particular we simplified things here
greatly by assuming that there are just two possible occupational statuses namely wi is
one if individual i works white color and it is zero sorry if individual i works blue color.
Now one could make an argument that earnings not only depend on a college degree but they also
depend on the occupational status given that people who work white color typically have higher
earnings than people who work blue color at least on average. So the question is how do we handle
this additional information? And we have distinguished two possible ways to proceed here
one which was discussed last lecture which used just the sub-sample information by using for
instance just the white color vercas and then estimating how for white color vercas the
the college education affects the earnings of these people. And what we came up with was then
the conclusion that it is not a good idea to just condition on this sub-sample because what
happened was scanned through my slides here that we found out that the effect we isolate is actually
a mixture of two effects namely the causal effect of a college degree for those people who work
white color wi is equal to one with a college degree so potential outcome of somebody who has
a college degree and works white color. Here the difference between the two potential outcomes in
terms of earnings is the actual causal effect we aim at and then this effect however is confounded
by a selection bias which consists of the difference of the potential earnings for people who do not
have college education so y0 here and y0 here by occupational status namely both of them work
white color but some have had a college education and others have not had a college
education so this type of selection bias also comes into play and confounds our assessment of
the true causal effect of the college degree. So the bottom line of this first case analysis
from the last lecture was that it is not a good idea to just use such a sub-sample and I think
this is perhaps not completely apparent to everybody when I first suggested this possibility
but using theory we see that it does not give the correct result. Now we want to come to the second
case today where we ask the question whether we can use the information on the occupational status
of each individual by conditioning on this variable in the regression so by really adding it
to the regression as a additional covariate and you see this idea from last year in equation 39
where we again regress the earnings y on constant on beta two times the dummy variable for college
education so just we regress on the college education estimate beta two so this is actually
our variable of interest and we also then regress according to this suggestion which would turn out
to be a bad suggestion on the occupational status of the the burgers and so on the dummy variable
which indicates either blue color or white color burgers with a coefficient of beta three. Now the
first thing if somebody suggests this equation here the first thing which you should recognize
is that this is not a saturated regression so that's easy to see what do we find we have here
three coefficients which we estimate beta one beta two and beta three but there are four different
situations on which we condition namely the two dummy variables can both take on either the
variables zero or one so there are four different combinations for these and if we evaluate equation
39 for those four different combinations then we see that well if ci is equal to zero and wi is equal
to zero then very clearly the expectation of yi is just the constant beta one and if ci is equal to
one and wi is equal to zero then the expectation of y is beta one plus beta two because we would
expect this we would expect this here if we condition on ci wi and conversely if we condition
on ci equal to zero and wi equal to one then the expectation would be beta one plus beta three
since this is one but the beta two would not be present because ci is equal to zero
and if both of them of both of the dummy variables take on the value one then of course we would
have as expectation beta one plus beta two plus beta three so that there are four equations and
three unknowns and that is not a set here etc so how would we interpret the coefficient on
c beta two now i give you now one possible interpretation of it there's another one which
you are asked to do in an exercise so beta two would be for instance equal to the expectation
of earnings given college education and blue color minus expectation of earnings
give no college education and blue color right so that essentially would be the effect which
we have if we just look at blue color markers wi is equal to zero in both cases and then we have
college education here and we have no college education there the expectation of the difference
of the expectations of the earnings here is equal to beta two okay so what does this lead to if we
follow up on on this idea of interpreting beta two well clearly if ci is equal to one then we
have college education so the observation we make on yi is actually the potential outcome y1i so we
can write y1i here and also we can write w1i here instead of just the observed value of wi
w1i is the potential occupational status of individual i in case that individual i has to
add a college education so this is why the one is here we can replace the observation by this
potential occupational status because we know we are currently looking at people which have
a college education so the observed value is actually the potential occupational status here
the same reasoning we can apply to the second term here this would be a term which is conditioned
on people who do not have college education ci is equal to zero so the observed earnings yi would be
equal to the potential earnings in the case of no college education and the observed occupational
status would be equal to the potential occupational status in the case of no college education
so this equality here transforms observed quantities into potential quantities well and
then we have made an assumption which i didn't mention in my production today but which i just
asked or defended in the last lecture namely that college education is randomly assigned which
usually is not the case in reality as i have emphasized but just for greater clarity here
or for easier set up here let us assume that college degrees are just handed out at random
to the the population which implies that the ci dummy variable whether somebody has a college
education or does not have college education is completely independent of the potential outcomes
y1 i y0 i w1 i w0 i which then means that by rule one of the rules on conditional expectations
in the review of probability we can just forget about the conditioning variable ci
in these conditional expectations here and write the same thing as the expectation of y1 i given
w1 is equal to zero minus the expectation of y0 i given wi w0 i is equal to zero so we just keep
the second part of the conditioning information here because the expectation is the same regardless
of whether we condition on ci or whether we do not condition on ci when this is then the same
problem as the one we have had already in the previous analysis in the last last lecture
that it doesn't isolate us the causal effect that we are chasing because here we have different
groups of people over which we take the expectation over who's earning we take the expectation
this year would be the expectation over earnings of those people which have an occupational blue
color status even though they have had a college education and this is over those people which have
blue colors that status without a college education so these are clearly different subgroups
of people and we don't know whether the expectation of y1 i compared to the expectation of y0 i
for potential outputs evaluated over different subgroups of the sample would be a fair comparison
so actually we can easily compute where the selection bias comes into play by adding and
subtracting the same quantity and thereby getting two different types of effects so we see we have
here the conditional expectation of y1 i we have the same thing here with the same conditioning
variable but then i subtract the conditional expectation of y0 i conditional on w0 i is equal
to zero and in order to make up for this here i sorry is there a mistake here yes i think so
i have to add the same expectation here and where's the error here
sorry for that
oh no sorry i was mistaken here i should have started with this term right so this term here
of course this term here is the expectation which i take here the expectation of negative
y0 i given w0 i is equal to zero is this expectation here which is taken from the
previous line of the set of equations here and i add the expectation of y1 i given y0 i is equal to
zero this expectation of y1 i given w0 i is equal to zero has been added so i have to subtract here
again right this term here this negative term makes up exactly for this new y1 i here which
allows me to draw a comparison between potential outputs for people who have blue color status and
no college education and this term here is the same term as this term there so nothing has changed
there basically what i did was i added this here and subtracted it here again to come up to a causal
effect which we have here for the same group of people now blue color workers without college
education i have here the expected difference between the potential outcomes in the case of
college education or no college education so that's a causal effect but the beta two not only
comprises this causal effect which we actually would like to identify but there's also a selection
bias which is the difference between these two terms here and you see these two terms refer to
the expectation of the same potential output y1 i here but conditional on different groups of people
namely here blue color workers which have college education and here blue color workers which have
no college education so this is a typical selection bias term here and beta two is the sum of the true
effect and a selection bias so beta two is affected affected by a selection bias
this is unfortunate so the beta two in this interpretation is not what we actually
look for because we have this confounding term which expresses the difference in potential earnings
with college education for people whose college whose potential occupation
is blue color with versus without college degree
now as an exercise as i have already announced use the same representation of beta two to show
that you may equally well write beta two as the expectation of y1 i minus y0 i for blue
color workers which have had college education plus a selection bias which would be the selection
bias which occurs when you compare earnings without college degree here and there for
different groups of people namely blue color workers with college education and blue color
workers without college education right so use the the model which which i gave to you to show
that beta two could also be interpreted this way just the same way as i have interpreted here
now there is actually there are actually two more interpretations of beta two and these
come from the fact that the regression is not saturated so i give you just one of those
representations and then on the next slide you will again get the exercise to find the fourth
interpretation so since we have an equation here which is not saturated we have this over
determinization of the beta coefficients four equations determine three beta coefficients so
they are already determined and one example one way to or what one alternative way to interpret
beta two would be this way here beta two is equal to the expectation of y i given college
education and white color minus expectation of y i given no college education and white color
right so the difference here just no college education here college education here in both
cases it would be white color why is this beta two well let's go back to this thing here
we basically take the difference of the fourth equation and the third equation right subtracting
the third equation from the fourth equation the beta one cancels and the beta three cancels so
we are left with a beta two and this as you can see is the expectation of y i given college
education and white color minus the expectation of y i given no college education and white color
precisely what i have written here now again we can rewrite this equation here in terms of
potential quantities so transform the observed values y i w i into their potential counterparts
which would then give y one i here because college education has been successfully completed here and
then of course the w i which we observe is the case where the individual is white color and has
had college education and we can kick out the conditional mean variable c i because we have
assumed that college degrees are just distributed randomly so the c i is actually independent of the
particular i had this in two separate steps in the previous interpretation here i just saved a
little space by doing just one step so we get this conditional expectation here minus then
this conditional expectation here where we take the expectation of potential output in the case
of no college education because this is what it is if we have no college education and the wi is
then of course also w zero i here which is equal to one so conditional conditional on w zero i
being equal to one we can take the expectation of y zero i here and well then again we identify
the causal effect as expectation of y one i minus y zero i given w one i is equal to one
and you see the expectation of y one i given w one i is equal to one well that's exactly this term here
whereas the expectation of y zero i given w one i is equal to one does not occur in this previous
line so i have just subtracted this thing here therefore i need to add it again so that the
equation holds that remains balanced so i add the same term here and then i subtract of course this
term which is precisely this term up here so that would again be the causal effect this is the
causal effect of college education now quantified for people which were quite color and have had
a college education this is the effect we are interested in even though it is not the same
effect as the one we have identified in this interpretation here however both of them are
confounded by selection biases and these are obviously different selection biases here we
have the selection bias between earnings in the case of no college education for individual i
here and there for two different groups of people namely w one i equal to one means
white collar workers or both cases these are white collar workers but once we have them
without college education and then here we have them with college education so here we may have
differential effects of differential effect in terms of the y zero i's here and that constitutes
a selection yeah so again we have the same type of problem as before and we would see that
which we estimate is not the causal effect we would like to find so here again the exercise
for the second representation of beta two show that beta two may alternatively be written in
this form here and then interpret both terms of the expression so this term here and that term here
summing up that's quite a mess right if we have a non-saturated equation and then we condition on
bad control then we get four different interpretations of beta two right and in no case is
the interpretation the one which we actually would like to have for for beta two it's this
no of these none of these interpretations confirms with our objective of identifying
the causal effect of education why do we have foreign interpretations well because we have
two problems one problem is that the beta two is over determined since the regression equation is
not saturated therefore we have two different interpretations of beta two i write here multiple
interpretations which is formally correct but perhaps it would be better to write that there
are two interpretations and then yeah okay the way we write it actually we get we get even four
interpretations so in that sense it's perhaps even better to write multiple as i have done here
rather than to write two and the second problem is that due to a selection bias which always comes
up regardless which of four interpretations we take we just do not recover the true causal effects
now the first problem we in principle at least we know how to deal with it we could specify
saturated regression rather than not saturated regression so let's see if a saturated regression
equation would also solve the second problem right so pretty sure that it would solve the
first problem so can be a good way let's see whether it would solve the second problem
how would we specify saturated regression equation well we would add a new regressor
z and this regressor z would be a product term between the two dummy variables so always c i
multiplied by w i which again is a zero one variable and would be zero everywhere except
where c and w are both one so that's a normal interaction term i would find then with this
saturated setup that if c i w i are both zero then all these terms cancel so the expectation of y i
would just be beta one whereas yeah that's correct whereas if c i is equal to one and w i
is equal to zero well then the z term is apparently equal to zero right because that's the problem the
product of c i and w i the z i would be equal to zero and we would have w i equal to zero so beta
three and beta four would not be part of the expectation rather the expectation would be beta
one plus beta two now we get this thing here and similarly if we have a c i equal to zero
w i equal to one then we have beta one plus beta three and if all four uh if all if both
diagrams are equal to one then we would get the sum of all four parameters so now we have four
equations and four parameters and clearly the regression equation is saturated the problem of
over determination is solved fortunately the saturated approach has unique interpretation
of beta two so how would we find beta two now well let's look at these equations uh here how can we
actually find beta two we see beta two is in the second equation and it is in the fourth equation
so one way to find beta two would be beta two or would be the second equation minus the first
equation right that should give beta two because the beta one would then cancel can we do something
similar here where we would need to have an equation which we can subtract such that beta one
beta three and beta four cancel to have beta one cancelled well that's easy we actually can use
equation two to subtract it from this equation and we would have beta one and beta two fallout
would be cancelled but who would still be left with beta three and beta four and there's no way
to get rid of the beta four here right uh so the only interpretation of beta two would be the
interpretation which we get when we subtract the first time from the second time which is uh what
i do here uh the first uh sorry the second line uh was um the expectation of why i given college
education and blue color status and the first line was the expectation of why i give no college
education blue color status again we can rewrite this in terms of potential um quantities here
we have college education as the status so this yi is actually y1i and this wi is actually w1i
here we have no college status or no college degree as uh the status so the yi would actually
be y0i and the wi would be y0i then again using the assumption on random assignment of college
degrees we can just cancel the ci variables from our conditioning variables here and know that uh
the expression which we have dealt with so far is the expectation of y1i given w1 is equal to zero
minus the expectation of y0i given w0i is equal equal to zero and then by the same type of techniques
that we have used already we see that the saturated approach does not solve problem two
because again we have two different groups here which we compare if we want to compare just
with reference to one particular group in this case why the w1i group so the group of people who
work uh blue color even though they have a college education we can of course write down the causal
effect we are interested in which is this one here but then we also have sexual bias and uh
this is exactly what was the problem already in the previous attempt so how does this come about
well we see again we condition here on w1i equal to zero so we condition this thing so we have
already the y1i here we have to subtract y0i which means we also have to add y0i given w1i
equal to zero so this term has been added to the equation to make up for negative y0i given w1i
equal to zero here and this thing then is of course the same thing as this term
that's one way to do it the other way to do it is that we take the y0i given w0i equal to zero
as given so take this term here as given which would be equal to y0i here given this condition
w0i is equal to zero rather than w1i be equal to zero as we have had before so here we have w0i
equal to zero and then in order to do a fair comparison we add the term expectation of y1i
given w0i is equal to zero and this expectation we subtract again here
well what is still not accounted for is the term y1i given w1i equal to zero which is this term
here and we then see again that we have a selection bias of this size here which is now the selection
bias for earnings with a college degree evaluated first for a group of people which works blue
color and has a college degree and second for a group of people which work blue color and do not
have a college degree which are not the same people again now the next exercise i have basically
already done for you but perhaps it's still useful that you do it again at home convince
yourself that there is no other representation of beta 2 from this equation 41 here from the system
of equations 41 yeah we have done that essentially already now you see that the saturated approach
does not solve our problem we wrote the beta 2 now in two different ways but in either way we
cannot isolate the causal effect which implies that even in saturated model we do not find help
against the bias which is induced here by bad control however what we can do in order to estimate
beta 2 is that we forget about this bad control altogether so what we can do is that we just
estimate the beta 2 in the simple regression where we regress y on a constant and on the college
variable c this regression 42 would be an unbiased estimate of the true unconditional causal effect
provided of course that the college degree is relatively assigned
so it may be that an unconditional comparison where we do not bother to include covariates
is very straightforward it doesn't give us any interpretational difficulties
whereas a conditional comparison may perhaps not tell us much that is actually useful for us
unless we also supply a more elaborate model of the links between college occupation and earnings
okay this concludes the discussion of case two let us now go to the issue of bad proxy controls
so essentially here we combine two scenarios or two issues which we have handled differently or
in the previous lectures one issue is the issue of a proxy variable which is included
to partially capture the effects of omitted factors and now we not only consider such a
proxy variable but also think about what happens if this proxy variable is actually bad proxy
variable bad proxy control because it is itself affected by the causal variable
we do now the same type of exercise again in the very simple version of the proxy control
scenario which we have dealt with earlier so assume that we are interested in the effects
of schooling on wages something like a school experiment like nc school experiment
ideally we would use long regression in this form 43 here where the earnings which we now
record are regressed on a constant with coefficient alpha and then on some measure of schooling
success of schooling and perhaps some measure of ability plus epsilon
now ability as we know is in general not easy to measure or usually we do not have data
ability but what we can think of is to use a proxy variable for ability and in order to
make things not too complicated i assume that here this a is already a proxy variable for
ability let's say an iq score now there are different possibilities what type of data we
may have at hand for the iq score one possibility would be the possibility that for instance we
have the iq score of children let's say in the eighth grade we assume that everybody
uh completes at least the eighth grade of high school and then let's say during eighth grade
the iq score of children is tested and we have uh what the iq the intelligence is tested we have
the iq score of children this is what this variable a is you may say then that the variable
a measures something like innate ability something which people just get from
genetics from their family from from from school from factors which are currently
outside of the relevant schooling choices because everybody completes at least eight eighth grade
so basically innate ability would account for ability which has been educated in schools but
this type of basic schooling is uh gone through by by everybody everybody goes through this type
of basic schooling so there are no choices made about education so far until eighth grade but
after eighth grade people start choosing some leave school after eighth grade others carry on to
a level let's say 12th grade class or something like this still others continue on to uh to
college obtain college degrees so these are then different levels of education which not everybody
achieves now the point is that if we interpret the s variable the schooling variable as schooling
after eighth grade right then the a as the iq score of children in eighth grade is a good control
right because it's determined before the schooling choice has been made by the children
but only if we say well s captures schooling after eighth grade
this implies that the error term in this equation which is an individual specific error term is
uncorrelated with the regressors so we would say the earnings are subject to certain errors epsilon i
and these errors epsilon i are uncorrelated with the s i because the errors occur only after
schooling choices have already been made and after school choices have already been made
and we're still interested in equation 43 and suppose now that unfortunately data on a as an
iq score which results from tests in eighth grade is unavailable the class hasn't been tested for
instance but suppose we have a second ability measure let's say a second iq score some type
of test results on intelligence collected later after schooling has been completed
further suppose that this is the score on let's say test used to screen job applicants
and call this variable then the late ability al so the ability which people achieve
only rather late in their life already when they argue young grown-ups
well in general we should expect that schooling does increase the late ability relative to the
innate ability so specifically let's suppose that the late ability al is a function of the innate
ability we would argue that the innate ability is the higher is also the late ability very
plausible and we would argue that the schooling the schooling choice after eighth grade also has
some influence on this late ability so probably we would expect that the coefficients pi one and pi
two are positive and then we have a concept somehow so schooling and innate ability increase
the late ability now we can um suppose that there is a researcher who worries about an
omitted variables bias and aggression where we regress the earnings just on s just on the
schooling variable he says well earnings not depend on former schooling earnings also depend
on ability so we should actually in order not to fall prey to some some omitted variables bias
we should also include a measure of ability and unfortunately the only measure of ability which
we have is the iq measured by al by the late ability we do not have the early ability here
so the researcher which is worried about the omitted variables bias now regresses y on s
of the schooling variable and on the late ability al because the desire to control the true ability
is unavailable so the not really true available ability but the iq in eighth grade is unavailable
assume that this here is the true model so we can use this now to substitute in for
the unavailable data a in the regression on s and al we have here y being regressed
or we tend to regress if we have the data available y on s and on a the innate ability
but the innate ability not being at our disposal we replace the a now by the al
and write this as al minus pi naught times uta minus pi one times the schooling variable
divided by pi two so essentially what we do is we solve this equation here for a and then we get
that a is equal to al minus these terms here and then divided by pi two
plus of course the error term epsilon here now what kind of effect does this have if we proceed
as i have done here well first thing is that we see the coefficients change the coefficient of
the constant changes from alpha to some alpha minus gamma pi naught divided by i2 because we
also have to take into account this term here moreover the coefficient on the schooling
variable also changes the coefficient on the schooling variable is actually rho and this is
the causal effect this is what we would like to estimate but we would estimate for as a coefficient
which is actually related to rho minus gamma pi one divided by i2 and then we have an additional
regressor al this is the one we have recorded and this additional regressor would have the
coefficient of gamma times pi two gamma divided by pi this coefficient itself is not so very
important the important thing here is that we do not estimate the true causal effect we do not
estimate the rho but rather we estimate rho minus gamma pi one divided by pi two
that's the estimate which we will get and this means that the estimates are biased unless either
gamma is equal to zero or pi one is equal to zero or formally you could also say pi two is equal to
infinity that this will never be the case so these two cases gamma equal to zero or pi one equal to
zero we would have the result that we estimate true causal effect but unfortunately this is not so
very likely what do these coefficients stand for well pi two is the coefficient let's first look at
pi two is the coefficient of a of the innate ability in the regression of the late ability
on schooling and innate ability so if this pi two is small then the bias will be large
if innate ability has only a minor impact on late ability and most of
late abilities formed by schooling which some people say is the case and they downscale or
downplay the importance of importance of family and genetic factors if this is the case then the
pi two would be equal to sorry the pi two which you see here would be rather low and now obviously
if pi two is small then pi two would be the result of that proxy so it would basically say
that al is not a good proxy for the missing observation a and therefore if we have a bad
proxy then the bias will be large right because we divide by pi two here and the greater
and the smaller pi two is the greater is the bias here
if pi one is equal to zero then schooling just doesn't affect late ability and hence there is
no bias because late ability would then essentially be the same as ability but if gamma is equal to
zero then ability does not affect wages and therefore the proxy variable does little harm
we have no bias but we have a higher variance of the estimates right gamma was the coefficient
which we had here right gamma is this coefficient here and this would say well the innate ability
just isn't important for the regression therefore we actually would not need a proxy variable we
could just run the regression which regresses y on a constant f of s but if we add this additional
regressor well then not much will happen except that the standard error is increasing
now in this model we would expect the bias to be negative and please think about it and tell
why we would expect that bias is negative what we conclude from our thought experiments here is
that the use of a proxy variable that is affected by the variable of interest generates biased
coefficient estimates so again we have a problem here in this case because the proxy variable
is not well chosen because the proxy variable is actually affected by the treatment that should
now be the case okay
while we use a proxy control this may bias the estimate of the regression coefficient
of interest but it may nevertheless be an improvement on the previous situation where
we had no control at all so no covariate at all right to see this note that the motivation for
proxy control is equation 43 which is this equation here which tells us that the true
equation actually requires some a and the proxy control variable had has had something to do
with this a so it captures some of the information which we lack because we do not observe the a
this is meant when i write here the motivation for proxy control is equation 43 now this is also
why i inform you that using a bad proxy bad proxy for the control variable has undesirable
effects in the sense that the estimate will become biased but it may also have positive
effects in the sense that the omitted variables bias is reduced and to see this look at the formula
for the omitted variable spines which we have derived in this lectures when we express this
omitted variable spice in terms of the parameters of the model then we have that the omitted variable
spice is equal to gamma times delta where the delta would be just the coefficient from a regression
of the omitted variable the so of the non-available variable a the ability variable the iq measured
in eighth grade on s we have no other controls except the constant
hence regressing the wages just on a constant and on s would be a regression which
suffers from an omitted variables bias and would therefore yield a coefficient estimate for s
with expected value rho plus gamma times delta this is the omitted variables bias
with a bad proxy on the other hand we would expect an estimate which is equal to rho minus gamma pi
one over pi two as we have derived in the equation 45 so not using the bad proxy would also not give
an unbiased estimate but would give us a biased estimate with rho plus gamma times delta
expectation or asymptotically with using a bad proxy would turn things around
but still result in bias this time it would be a negative bias here it is a positive bias
so essentially we have two possibilities to go wrong either we suffer from omitted variables
bias or we suffer from bias of bad proxy control now it may be of course that the bias with bad
proxy is absolutely smaller than the omitted variables bias and it may be exactly the opposite
and it may be exactly the opposite maybe that the omitted variables bias is absolutely smaller
than the bad proxy so it all depends on the magnitudes of this delta coefficient
versus the ratio of pi one over pi two
so it can be worthwhile to use both approaches just to see how things work out
because if one bias is positive and the other bias is negative as it should be the case in this
example here then the true causal effect may very well lie between the two estimates
so we can sort of still try to give readers a feeling on where the true causal effect lies
we can say well we have one estimate which is more or less let's say upper bound because
the bias is positive and when we have one estimate which is more or less lower bound for
the true causal effect because here the bias is negative unfortunately these are not really
reliable bounds which we have here but it is more some kind of intuitive application to
communicate there then because it is not guaranteed that true effect lies in between
these two estimates even if the biases have opposite sign and as an exercise please think
about the question why this cannot be taken for granted that even if the biases have opposite
sign the true effect lies in between the two estimates
going away from this example and more phrasing it now in terms of general advice on controls
always consider and think of the timing which we have variables which are determined before
the treatment are generally very good controls so very variables which pre-date the treatment
cannot be influenced by the treatment so such variables are generally good controls
since they cannot be affected by the treatment when the timing is uncertain which also can occur
or unknown an unbiased inference on causal effects requires that we make explicit assumptions
about what happened first or it requires the assertion that none of the control variables
are themselves caused by the regressor of interest which is an unproven assertion
at first unless you find some arguments which support this assertion or even a proof
which supports a claim
including irrelevant control variables does not bias the estimates as long as those
in irrelevant control variants are not themselves affected by the treatment which I should perhaps
note in this in this sentence here but the irrelevant control variables of course do what
regressors which are irrelevant always do they increase the variance so in this case we have
then a trade-off between bias reduction and variance reduction we have a bias variance
trade-off if we do not control if we do not include the control variables then maybe the case
that we have an omitted variables bias but if we do include them and they are actually not needed
well then we would not approve on the bias or there would perhaps be just no bias because
there's no limited variables bias but we would have a higher variance in this case
for this reason it is often not clear a priority which control variables should be included but
thinking about a particular problem one can advance arguments which support a particular choice of
control variables so you should always justify why you include certain variables or why you do not
include a variable which some other guys have perhaps suggested may be useful and in order
to convince your audience either the professor you write your master's thesis with or later the
referees of a scientific journal it is always useful to provide regression outputs for different
sets of control variables so that people can check if something changes greatly when we change
the number of control variables i haven't introduced you to the causal graphs so we just
skip this and come to your research question again remember the research question which you were to
choose from a long list of research questions which i gave you for your research question
think about what are actually good control variables in your specific case in your specific
research question and which variables are perhaps bad controls which variables should you not
control for because they may be affected by treatment and are there variables which are
potentially omitted or which you have to omit because usually you wouldn't find data on this
so just go through your particular research question as a thought experiment again and try
to apply your research question to the questions we have addressed here in this section on control
variables okay and we move on now to matching and in particular to propensity score
matching what this is will i hope become clear to you very soon and
actually what you perhaps should first consider is the fact that so far we have just talked about
regression based techniques in particular we have thought about the question when the least squares
estimator ordinary least squares estimator would be a good estimator or when it would be either
biased or inconsistent now one thing we have never really addressed is the question of the
correct functional form quite mechanically we have in this lecture and most researchers also
do in scientific research assume that the functional form is linear right this is very
popular to use linear functional forms because you can always justify this as a first order
Taylor approximation which always gives you a linear functional form
but what if the functional form is not linear if we make an error in specifying a linear relationship
whereas the true relationship is actually non-linear possibly even highly non-linear we have
no knowledge about the functional form so in this case the least squares regression which is typically
applied to a linear regression equation can be misspecified and therefore the coefficient
estimates can be biased in some sense this is very similar to a normative variable if we ignore
higher order terms because as you know from the theory of Taylor approximations you can of course
compute the first order Taylor approximation and then you have a linear function which is what we
typically estimate with least squares estimation but we can also expand the the function to the
second or higher orders and then we would get further variables we would get squared variables
interaction terms with the second approximation already or second order approximation already
and even more so with higher order approximations so essentially when we estimate a linear
relationship we assume that higher order terms squared terms product terms interaction terms
all these things that they all don't play a role and if actually they do play a role then we
really have a problem of an omitted variable so in that sense specifying the wrong functional form
is actually a special case of an omitted variables problem now we do not actually need
to do OLS estimation when we want to do causal analysis there are other techniques and while
they are less often used they may be quite fruitful and I want to introduce you at least
briefly to these type of techniques one of which is matching matching is in some sense related to
least squares regression you can actually write down matching estimators also as least squares
estimators but that is highly technical and I won't present it in the lecture here I rather want
to explain the general principle of matching because the advantage of matching is that in
matching exercises we do not need to specify the exact functional relationship which ties x to y
for instance so in that sense I write here that storytelling is slightly different between matching
and regression because we do not need to justify a linear regression equation
so what is the general idea of matching the general idea is that when we look at a particular
unit or a particular individual which has been treated we try to find something which is called
a statistical twin so we try to find somebody who has the same observable characteristic
but has not been treated so somebody who is a member of the control group right so basically
matching means we take just one unit one individual from the treatment group and we look at it and see
what do we know about this unit about this individual what kind of characteristics does it
have what information is tied to this particular individual and then we search everywhere in the
control group whether there is a unit an individual which has basically the same characteristics
and then we match the two and say we compare the effect on one with the effect on the other
assuming that they are virtually identical right so in some sense this twin which we find in the
control group donates a counterfactual potential outcome y0 for the treated individual
this is an attempt to copy the idea of a randomized controlled trial clearly here we would have
treatment groups and control groups which are not necessarily rct but we can try to do it that way
by saying well if we find two units which are actually the same in respect of everything
except for the treatment well then this is as good as randomized right it's not literally random
but it is as good as randomized or at least we assume that it is as good as randomized conditional
on the observable characteristics which we typically collect in some matrix x
so here i come up with an example which is actually a bad example but perhaps it serves
its purpose nevertheless and after i've explained the example i will explain to you why i think it
is a bad example suppose we study the effect of unemployment on health there may be quite a bit
of an effect of unemployment on health people who are unemployed have little income right so
perhaps they have less medical possibilities at their disposal perhaps they even shy away
from visiting the doctor if their health insurance is expensive or they have opted for a rather
inexpensive tariff in their health insurance perhaps people who are unemployed start neglecting
themselves they may be depressed they don't care anymore they don't exercise for instance
perhaps they don't have the financial means to buy food which is healthy or they have some other
inclination to go for food which is rather not so healthy and makes them fat and perhaps lazy
again perhaps they start drinking and they develop alcoholism and alcoholism is an
illness itself but also is harmful to to your general health so there are many possibilities
why somebody who is unemployed may have had may have due to unemployment a health status which
is inferior or at least different to the health status of an employed person so what could we do
in order to study the effect of unemployment on health suppose for instance we look at people
who are unemployed the unemployed are now the treatment group right the treatment which the
people receive in this case is unemployed and suppose we take some person from the pool of
unemployed people person a say is female has a university degree is 42 years old and lives in
East Germany and we know what her health status is let's say it's 82 points of whatever this
this means so in matching what we would do is that we search for an employed person who is
female has a university degree is 42 years old and lives in East Germany and if our sample is
sufficiently large then we have a good chance of finding such a person perhaps we even find more
than just one then perhaps we should draw just more than one statistical twin but let's suppose
we just find one okay then this employed person which has the same observable characteristics
female university degree 43 years old lives in East Germany is the statistical twin which
generates the counterfactual suppose this second person person b has health of 95 points
well then we can compare the health status of person b with the health status of person a
arguing that the two are twins statistical twins and differ only by employment status
unemployed versus employed so that we could say provided the conditional independence assumption
holds that the unemployment the the reduction of unemployment reduces the health of the person by
13 points from 95 to 82 right we have 82 here 95 here so the difference is 13 points
this would just be the result for one single individual but we repeat the matching procedure
now for all the treated individuals so for each and every treated individual we try to identify
a statistical twin and after we've done this for all the treated people and ideally we found good
statistical twins for everybody we can compute the average over all these differences which in the
first case were 13 points at which probably are different for other statistical twins which we
match if we average then all these results then we have an estimator of the ATT that's the general
procedure of matching and please as an exercise just discuss why is this the ATT why isn't it
the ATN T or why isn't it the ATE
now it is not always as easy as i have now explained it is because when we have either many
covariates many x variables or we have continuous x variables then it is very hard to find an exact
statistical twin so for these observable characteristics it was perhaps quite easy to
find a statistical twin there are many people who are 42 years old there are many people who
have university degrees or live in east Germany so that's easy to find somebody who matches this
person exactly on those dimensions but now suppose that we have more information
on the person which actually is a good thing to have more information on that
suppose that the conditional independence assumption only holds when we also condition
on this additional information like for instance income now income is basically a continuous
variable you can have an income of 2 100 euros or you can have an income of 2 101 euros and that's
not the same of course you can then try to make categories but categories are also always different
because sometimes you put incomes in one category which are farther apart from each other than
incomes from two different categories like for instance if you say well one category is income
below 2000 euros and the other category is income between 2000 and 3000 euros then you would say
somebody who has 2001 euros of income is different from somebody who has 1999 euros of income even
though the difference is just two euros whereas you would say someone who has 2001 euros of income
is the same statistically as a statistical twin to somebody who has
2999 euros of income which is different than of 998 so clearly categories are not so very helpful
when we have continuous variables but with continuous variables and let's say a variable
like income it can be very difficult to find somebody who has exactly the same income
up to the last euro or the last euro cent or if we don't only know the
the education status in terms of has a university degree or doesn't have a university degree
but rather we know the major of the university studies there is somebody who has specialized
on far east philosophy for instance and then find somebody else who has specialized on far east
philosophy and is also female and 42 years old and has the same income or whatever body mass index
would be another indicator which is perhaps interesting but perhaps also not for what reason
i would like you to think about body mass index is actually a bad example or you can take
characteristics of the spouse so if we have many of such variables it is very likely that
no one in the control group of our sample has exactly the same characteristics with respect
to all the covariates so with respect to all the conditioning condition variables
which would imply that no exact statistical twin is available
the solution to this problem is or one of the possible solutions to this problem is the so
called propensity score matching which is due to Rosenbaum and Rubin already an older paper but
the fact that i still cited is indicative of the fact that the technique has found wide acceptance
so people still use this idea today and the idea is very simple you can try or well one way to
proceed would be to reduce the k-dimensional vector of controls which you have collected in some x
matrix to a one-dimensional scalar which is a certain type of probability right
so basically when we have many covariates let's say we have k covariates and these k covariates
or the k is big number so we have many different covariates and some of them are possibly even
continuous variables and we can try to reduce the complexity of the problem to something
more easy to assess in this case we will try to reduce to one-dimensional scalar
and in order to do this we may use the probability of a treatment d
given that x takes on a certain value of small x right we can define p of x as the probability
of treatment given that our covariates x our conditioning variables have a just a particular
value of x now this would mean that we compute here the probability of our of the treatment
given or or depending on the realization of the covariates and essentially what is done
is that we use this probability as the aggregator for our controls in the matching exercise
and call this then the propensity score matching because what we measure with this p here is
simply the propensity of an individual so the likelihood of an individual to undergo treatment
given his or her characteristics
when we do exact matching the identification of the causal effect is an implication of the
conditional independence assumption right which we would write as the expectation of y zero given
di is equal to one and x is equal to small x it's the same thing as the expectation of y zero given
d is equal to zero so non-treatment and x is given to and x is equal to small x
in the propensity score matching where it is perhaps too complicated to find this type of
statistical twins let's see that this formula basically describes the idea of statistical twins
because we would like to match people which have characteristics small x here and there
uh from the treatment group d equal to one to the non-treatment group so from the treatment
group we pick somebody who has characteristics x equal to small x and we would like to find from
the non-treatment group from the control group somebody who has the same characteristics x equal
to x and perhaps we just don't succeed if we do find this then everything is fine and then we can
take the expectation of those y zeros there or compute averages we have sufficiently many such
matched pairs but if we do not do this then the recommendation for a propensity score matching is
to do the same thing as i have explained here just for the p of x's so match somebody from
the treatment group b equal to one with a specific likelihood of being treated
with somebody of the non-treatment group who has had the same likelihood of being treated
well if the likelihood of having been treated is the same for these two individuals here then we
can say that the treatment has been assigned as good as randomly right and we have something which
is very close to random assignment and then we can make a fair comparison of the expectation
of the potential output of say y zero in the case of non-treatment right or in the case of treatment
whatever we want to look at essentially we try to eliminate the selection bias when we condition
on individuals which have the same propensity score
how does one estimate the potential propensity score this is something which i have to defer
to later lectures more specialized lectures in econometrics typically we would use models which
are called probit models or logit models and it or in tubit uh models um it may be these models
you may have already encountered in other graduate econometrics sometimes the general idea is
communicated in uh in other graduate econometrics but i won't cover probit and logit in this
lecture here um i can just briefly explain what it is all about basically it's a type of
regression technique where you assure um where you ensure that the dependent variable
stays bounded by the lower bound zero the upper bound one because we want to estimate um some
relationship which yields as the dependent variable the probability and a probability
should always be between zero and one right so in a linear regression the dependent variable can
basically take on any value in particular it can be a value far outside the interval
zero and one and this we would not be able to interpret as a probability
so this is why the propensity scores must be estimated by uh probit and logit and you can
employ maximum likelihood techniques then which i also have not covered in this lecture
when we have estimated the propensity scores using say probit models or logit models then
for each treated unit we try to find the nearest neighbor in the control group right and there are
various methods i will come to this in the next uh slide and various methods to find the nearest
neighbor or even the nearest neighbors but the criterion for being in the neighborhood of being
the nearest neighbor is then the uh propensity score so it's the probability of treatment
after having identified these statistical twins in step two it is highly recommended that you check
the covariance balance between the treatment and the control group after matching so check
whether actually for the original covariates x which were too high dimensional to find exactly
the to find exact statistical twins for check whether these covariance have at least approximately
the same distribution in the control group and in the treatment group this is called check checking
the covariance balance because if this is not the case then it is quite likely that we still have
selection bias but if we confirm that there is at least approximated balance in the covariance
then we can with some confidence assume that the neighbors nearest neighbors the statistical twins
which have been identified donut as something like a counterfactual outcome and using these
counterfactual outcomes then of course one can compute the att as the average over all the
individual treatment right so for each individual we would compute then delta is the difference
between the potential outcomes and then averaging over all those deltas this would actually be
probably delta i we estimate then an aggregate delta
how do we choose the nearest neighbors well there are many many different possibilities
what i have explained to you here as the simplest way is just take the nearest neighbor apply some
metric which measures distance and then take the nearest neighbor but people have sometimes argued
that perhaps one should not take just one neighbor but perhaps a couple of neighbors so for instance
we fix some k let's say k is three and we take the three nearest neighbors to one person which
has been treated or that may perhaps make inference a little bit more robust or other
people have argued that we do not fix the number of neighbors we don't say three neighbors because
there may be some observations which do not have three neighbors in a closed neighborhood and then
perhaps it can occur that we would take some observation as a neighbor which is actually far
off even though it is still the nearest neighbor but it may be far off of the of the original
in terms of its characteristics may far off of the original observation that we want to match
so in that case then the k nearest neighbors matching is not a good idea so people have
also advocated radius matching where you would take the treated observation and then you define
in your metric some radius around the treated observation and then in the donor pool you look
whether there is anybody in the donor pool in the control group donor pool is the same thing as the
control group whether in the control group there is any observation which falls in this ball which
is described by by the radius and this can be zero observations can be one two three four five
so the number of neighbors is then not fixed of the radius fixed
or you can define some kernel so some distribution over the characteristic and then say that well
within one or two standard deviations of the distribution of some some other degree of standard
deviation point five standard deviations if you want to keep it close you take all the observation
in the the control group which shall be matched to the treated unit and you again do this for
every treated unit and then there are informational strategies which give you optimal matching and
so forth so lots of different strategies have been developed in the literature but all these
techniques have one thing in common they are based on the same identifying assumption on the same
identification namely on the conditional independence assumption yeah um often it is
the case that the differences between these different techniques which i have now just
briefly characterized are not very large um so um it can be a reasonable approach to say
we try them on the software available to do all of that so it's not a great effort to do it with
different approaches and then choose the variant which has the best covariate balance
okay uh your research question again um think about how would the propensity score equation
look like in your case so how would you try to estimate the probabilities of being treated
given a certain range of of covariates now for the reason readings matching and regressions for
the next lecture let me just go back because i have forgot i forgot to tell you about it
why i actually think that this example the effect of unemployment on health is a bad example
and and this is simply because a causality can move here both ways uh what i have explained
was that unemployment can have an effect on health but very clearly health can also have
an effect on unemployment somebody who isn't healthy is probably much more likely to be
unemployed and if we have causality moving both ways then of course it is much more difficult
to find the correct causality just one way just one direction right so um the effect of
unemployment on health is a particularly tricky case to to treat and i would not like to claim
that you can treat it in exactly the way in which i have presented it here um but perhaps you found
nevertheless illustrative how i presented it and could understand the principle of matching
well so then it has certain benefits it's not a recommendation to actually
quantify the effect of unemployment on health that way
all right this is uh that oh yes and matching and regression actually i think i will not
cover um because uh this is just what i have already said that you can write matching estimators
also in regression form but that is technical rather newer development i think i yeah i cited
the black paper uh here from 2015 if you're interested in it then read it i won't ask you
about that in the final exam so i just noted here uh that matching and regression have more in
common than perhaps you initially think and that's it then for this lecture and we continue
with the seventh set of slides in the next lecture