Thank you.
Okay.
Thank you.
Welcome, everybody, to the University of Hamburg, to the master's program in economics of the economics department here and in particular, of course, welcome to this lecture
of econometrics estimation and inference in econometrics.
I hope that all of you have settled well in Hamburg. As far as you have been able to travel to Hamburg.
I know, however, that some of you have not yet been able to gain a visa for Germany and will follow this lecture from abroad, which is one reason by the way why I decided to give the lecture in the digital format.
Another reason is, of course, that the corona virus pandemic is still with us and we don't really know how well we are currently protected and how the infections will evolve now when the weather is getting colder.
There was previous advice actually by the president of the university to stick to digital format with smaller lectures, which I do so also with smaller lectures, which I do here.
This implies that we cannot communicate in the usual way with each other, but the format which has been developed by the university allows for interaction via chat.
And I would like to encourage you to pose your questions via the chat function please let me point out that there are actually two ways to communicate with me during the lecture, either using Q&A or using the chat function.
It's always a little bit annoying if some comments or questions come in via one channel and the others come in via the other channel so my request is that all of you just use the chat and do not use the Q&A function this makes it somewhat easier to handle this.
For example, I will then read out the question for data protection reasons you're not allowed to speak to me in the recorded lectures.
So I will read out the question then and then I will answer the question. And as I say, please use the Q&A. Use the chat function and not the Q&A.
I'm happy to welcome my post talk Mr. Saeed Khuragarian, who you probably can see on your screen.
Saeed is going to give a parallel lecture which is called methods of econometric analysis. This is not a tutorial in the usual sense of the word as you probably are used to tutorials in the course of your bachelor studies, but it is technically a separate lecture.
However, we have arranged that the lecture of Mr. Khuragarian will mirror contents of my lecture and will in particular put emphasis on applications of the theory which I teach in this lecture here.
And this is said, I would first like to hand over the floor to Saeed Khuragarian so that he can introduce himself and introduce his lecture methods of econometric analysis.
Yes, thanks so far. Also from my side, welcome back or welcome to the University of Hamburg, so wherever you are at least enjoying us online.
Our meeting will take place on Wednesday, so you know the schedules that will not have a regular, so will not have a weekly meeting, but a bi-weekly meeting. So we start this week on Wednesday at two o'clock.
I've already set up a first problem set which is available online on the Steener webpage and I also send an email to you as far as you are registered on the Steener webpage with the login information.
So my suggestion is that I'm not taking much time from Mr. Lukas' presentation here, so I would suggest that we postpone our discussion to Wednesday, we will have 90 minutes on Wednesday.
I will give a short introduction on the ideas, how the setting of the class is, and if the format over there will be very similar to here, probably it's a little bit more informal.
We can also keep some space for kind of an, I don't want to call it personal exchange because it's not personal, but at least giving you some options for a Q&A if something is unclear in the lecture or if you want to put more emphasis on some certain aspects which are very hard to understand in the lecture.
And I will try to, I will try my best to come up with easy examples, maybe providing some numbers that we can do some calculations, ISO on hand or in Excel, I will keep it very easy so you can understand the topics over here much more.
So the purpose of my part is basically to support the knowledge which you gain from here.
I guess that's actually my part.
I think so. I'm not sure, are there any questions relating to the information just supplied by Mr. Holoverdian?
If so, if you want to raise a question, then please first raise your hand so that we know there is a question coming in via the chat, then you have some time to type it in.
When it comes, then we read out. It seems there is a question.
The first problem set is already uploaded in the STINA webpage.
It is important that all of you have access to STINA. This is usually the case, but sometimes there are students who have registered for this class from different departments or different programs, and if this brings with it that you are not properly
registered in STINA or don't have access to STINA, then please talk to the student's office and arrange for you being admitted to STINA in the regular way, because we will always communicate all the information via STINA.
You will find all the slides there, you will find all the information on syllabus and organization of the lecture there, so it is essential that you have access to STINA.
Oh, of course, sorry. Yeah, so my lecture is on Thursday. We start on too sharp on Thursday. I'm sorry, I guess I said Wednesday, I have a different class on Wednesday. Yes, so to clarify, it is on Thursday.
Okay, good. That's very good that you check that directly. So the lecture is on Thursday indeed. Can you just tell me the time again?
We start at two o'clock sharp.
Two o'clock sharp. Okay, great. Any other questions?
Good. Then I think that's that. Thank you very much, Saeed, for your presence here.
Have a nice lecture. Enjoy your day. Thanks. Bye-bye.
And then I will then continue with some remarks on the organization and the content of this lecture here.
I'm going to start with organization. You know, the lecture takes place today, so some Mondays and Tuesdays. There are two blocks of 90 minutes lecture on each day.
One block of 90 minutes. So we have 180 minutes altogether in a week, which means that this is quite a lot of material, which I'm going to cover in the lecture.
The lecture will finish with two written exams, one in February and one probably at the end of March. You are free to choose either the first exam date or the second exam date.
If you are unlucky and don't pass the first exam, then you have the possibility of repeating the exam in the second exam date.
If you just go for the second exam date and you run into the misfortune of not passing the exam, then your first chance of redoing the exam would be next year. This is something that you have to bear in mind, please.
Which does not mean that I advise you to necessarily take the first exam date. It depends, of course, on also the course load you have with other classes in this program.
It certainly does make sense to distribute the exams between the first date and the second date so that you don't have to have all the material readily prepared already in February.
So it can make quite some sense to have some of the exams pushed to the second exam date, but this has, as I said, the drawback that a possible repetition of the exam would require almost a year before you can redo it.
And this you have to somehow balance out how you want to take the exams. And I hope that the exams can be taken in in person form. This was not possible last year, which led to my postponing the exams until we could take it in person.
I generally do not hand out online exams, so no digital exams or take home exams or something like that. But the exams here will be an in-person exam and only other very exceptional circumstances, meaning personal hardships which are not within the responsibility of the concerned student.
Do I offer a possibility to take an exam online at the same time when the whole class takes the exam in person? So this may perhaps apply to people who have not been able to get their visa even by February, then we will provide for such a possibility.
But that is really a very extraordinary exception and you should prepare yourself for an in-person exam of about 90 minutes.
In the exams, we will provide for format, which is, I would say, 50% open book in the sense that I require student contributions for this lecture here.
Student contributions consist of a one-page handwritten summary of the lectures in a particular week. So both lectures, Monday lecture and Tuesday lecture taken together, you have the possibility of writing your personal notes on the contents of this lecture on a one-page,
on one page in handwritten form. You will have to hand this in by next week, Monday, so the Monday following the week in which the lectures have been given.
We check whether this is indeed a summary of the content that I have covered. So it must be in line with what I have actually taught in the lecture. You cannot just make a summary of what is written in the slides because
you never know exactly what I will do in the slides or by how much I progress and whether I cover everything which you perhaps anticipate that I covered. So it must be the content of the lecture.
And we will take that, whether it is the content of the lecture. If so, then the student contribution is approved and you can use it in the exam, provided that you have submitted at least nine such student contributions.
As you know, there are 14 weeks of the semester. We do not require a student contribution for the lectures of the first week, so for the lectures of this week, today and tomorrow.
We do not require a student contribution for the lectures of the last week of the semester. So it remains 12 weeks for which you can hand in a student contribution and you can hand in them only for these 12 weeks.
So for the first week and the last week, it's not possible to hand in the student contribution. From these 12 weeks, you have to hand in student contributions for at least nine weeks.
If you suffer from some kind of illness, you may get an extension for handing in the student contribution. In this case, please write to us, send us a doctor's
certificate digitally and then you'll get an extension for that. If you have other reasons not to submit a student contribution, overload or work or whatever, you can skip the student contributions three times, but you're not then allowed to hand them in at a later date and you can't use them in the final exam.
We do not grade or correct the student contributions, so we do not check whether what you have written down there is correct. So this is your responsibility that the student contributions are written in such a way that the content is correct and that you can actually use of them usefully in the final exam.
I just noticed this morning a mistake in the file that has been uploaded to SEMA, which contains the description for how to hand in the student contributions and how to submit them because the file has not been updated in this part to this semester.
It is actually still the information for the lecture last year. Sorry for that. So the particular dates which are given in this file are incorrect.
I think it describes the first student contribution is due some dates in November, which is not the correct date. We will correct this and upload the corrected file probably today. So if you download it tomorrow, then this will be the correct file.
Delete the file which is currently on SEMA. Please. Actually, I wanted to correct already prior to this lecture, but SEMA was unavailable for some reason. I don't know exactly why I could not log in and therefore I couldn't upload the corrected file.
There's a question coming in. Are we allowed to take notes handwritten but digitally? No, sorry. I don't quite understand the question, but perhaps it means that are you allowed to take the notes digitally rather than handwritten? No, you're not.
But for the simple reason that digital notes would be easy to copy and communicate to somebody else. So you're just allowed to submit handwritten notes, but you can submit them digitally.
This is actually the way we provide for. Take a copy of photograph or whatever scan of your handwritten notes and submit it then electronically to the email address which my secretary has established for that.
So you're not required to hand in actual paper, just the scan of the photograph of your handwritten notes.
And this will then be checked by us and my secretary will upload a file, an Excel file each week from which you can see whether your student contribution has been accepted as a valid student contribution.
In rare cases, and it is really rare, we do reject a student contribution because we say this is not the content which I have covered in the lecture.
Or it does not fulfill the minimum requirements and I didn't say that yet that you are allowed to submit one handwritten page, but it must be at least half a handwritten page. So if it's just three words, we would also reject it and say this is not a student contribution.
The purpose of the student contributions is actually twofold. One purpose I already mentioned, I think it is useful that you have some reference at hand when you go into the written exam.
Because typically, I mean, later on when you do work you also have the possibility to consult books and any kind of material and papers and everything so nobody has everything in every formula always in its head.
So it just doesn't make sense that you learn certain formula by heart, just for the purpose of this particular exam.
Even though I think it doesn't make sense to know some formula by heart, you should learn the most basic formula so that you can also reproduce them without the notes.
But for the exams, I don't care, you may write down any kind of formula or any kind of note which you want in your student contribution and you can then make use of it.
So I want the exam to be an exam which tests your possibilities or ability to apply what we have learned here, but not necessarily to show that you have learned everything by heart, perhaps even without understanding.
So it's more my intention to test how great your understanding of the material covered in class is rather than test how much you can learn about.
So, this is the one purpose of submitting the student contributions are basically forcing you to submit a student contribution because not sure if I said this already clearly enough.
If you do not submit at least nine student contributions you will not be admitted to the final exams. So mine is the minimum, you have to submit nine student contributions.
So the second purpose is actually to ensure that you continually work in this lecture, because, as a matter of experience, we know it is often the case that students during the course of the semester attend the lectures, but do not do much more
before the preparation of the exams really imminent, let's say, two weeks prior to the exam they suddenly start working through the material of the lecture and preparing for the lecture.
And I think this is not a good way to study, because if you do not rework at home at your desk through what I have done in lectures in a week, then it will become increasingly difficult for you to understand what I'm talking about in follow up lectures.
So it's really necessary that you go through the material which I have covered and understand what I have said. So I want to force you each week to sit down, look at the material which I have covered, take your notes, write down on a sheet of paper what you think is important,
what you may need for the final exam, and thereby by writing down by summarizing what I have done, also making clear to you what I've been talking about and what kind of learning experience you should have had from lecture.
This will prepare you much better for the lectures in the subsequent week. So this is the reason why I choose this format of student contributions.
As far as exercises are concerned, I will provide you with a lot of exercises in the lecture. These are sometimes exercises of a theoretical nature where you are asked to prove a certain statement.
Sometimes these are exercises which refer to data issues, but most of the data issues will actually be handled by, say, Khodavarian in his accompanying lecture.
And sometimes these are also programming exercises because I think that when you really sit down at a computer and try to program something in a mathematical programming language, you're free to use whatever language you're acquainted with.
So you can use MATLAB, you can use R, you can use Gauss, you can use Stata, whatever you find convenient, then your learning experience of what is actually done by certain econometric methods will be increased by quite a bit.
And sometimes it's also useful to run certain programs which do simulations to see how well certain estimation procedures, for instance, do with, let's say, artificial data and where problems arise from certain features of the data.
For this purpose, it is sensible to use artificial data because we can then design the features of the data exactly the way in which we want them to be in order to point out that certain problems are solved or are not solved by certain econometric techniques.
Yeah, so it is, well, a possibility for you and I would actually like to encourage you to solve these exercises at home and find the solution.
And I have solutions for all the exercises, but I do not supply them.
So I will not hand out any solution sheets for the exercises because the experience there again is that if I hand out solution sheets, many students just don't think a lot about the exercises or give up too early and rather wait till the solution is there.
But the learning experience, which you should have is actually the experience of finding yourself solution to some problem.
And this is why, as I said, I do not supply solutions to the exercise which I give.
But all of those exercises, of course, have well defined solutions and you should, when you understood what I've done in the lecture, be able to find these solutions.
So if you are unable to find them, this indicates to you that you have probably not yet mastered to a sufficient degree, a certain part of the lecture.
And I think this is all I wanted to say about the organization of the lecture. Let me now also say something regarding the content of the lecture.
All of you, or I would say at least most of you have had a decent econometrics education in their undergraduate studies.
Still, again, the experience we have on the master level is that much of that what you have learned in other graduates is shaky.
Or parts of it has been forgotten because preoccupied with other content of the undergraduate studies.
So when non-econometrics lectures, we allude to econometric methods, we often find that what should actually already be known from the undergraduate education
is not really well known, not well understood and students are unable to apply the methods problem.
This is a general problem which is not just related to this master program. I also had this many other teaching contexts.
And therefore, this lecture here, which actually should be a lecture on advanced econometrics, is in its first part basically just a repetition of what you should already know from your undergraduate lectures, both in statistics and in econometrics.
It is probably at a slightly more formal level than you have been taught these things in statistics and econometrics in your undergraduate studies.
This depends, of course, on where you had your undergraduate education. So you may find it a little more formal.
But the actual content of what I do in about two thirds of the lecture is just repeating what we would actually hope that you have learned it already in undergraduate studies.
Still, we found that is quite good to repeat all that and that there are gaps in knowledge of incoming masters students.
Only the last third of the lecture is probably new material to you. The last third deals with methods to establish causality or inference on causality in econometrics.
You probably know that one critique of many econometric methods is that econometrics just establishes correlations between data.
But does not allow to draw inference on actual causality.
And in the last 20 years or so, quite a few methods have been developed which claim that they can actually establish causality rather than just correlation.
And we'll cover, well, quite a few of those methods. It will also be more a survey of such methods than an in-depth coverage of such messages, such methods, excuse me, but these methods are newer than the standard material which we teach in econometrics
or it may well be that you're unacquainted with this method and have not yet covered them in your undergraduate econometrics education.
Yeah, but this is, as I said, just for perhaps the last four to five weeks of this semester that we will cover these methods.
And the first two thirds of the semester I will first present you with a brief review of probability, which you see already here on my screen, followed by a brief review of certain statistical techniques basically testing techniques, and then a rather comprehensive survey of econometric
technology. So the third part and quite a large part of the semester will be just a review of basic econometrics. And then from chapter four onwards we start with methods which allow us to draw inference on causality relationships.
Now I see a question here, would it be possible to upload the solutions at the very end of the semester. So we can make sure we got it right before the exam.
No, I do not upload the solutions even at the end of the semester, excuse me for rejecting that because this would have the effect that in the next year all the solutions would be known.
And the next year students would have exactly the situation which I described before that solutions are easy to get and little effort would perhaps be invested in resolving the exercises.
If you want to think, however, that your solutions are correct, you can at any time either ask Mr. Kolavergen to cover some of that in your lectures, or come to me and ask me, well, did I do this correctly.
I will I must say that directly not give sort of some kind of private tutorial than for students who would like to discuss the particular solutions with me.
And I will not tell them how to do certain things, but I will talk to them about the methods which have been applied in this particular problem and point out where perhaps a method has not yet been correctly understood
and not correctly being applied to the problem at hand.
So my so called office hours which don't take place in an office but take place digitally and by time arrangement would be also a place to ask me about the problems, the exercise that I have supplied you with in the lecture.
If you want to do this, I have one request to you, namely that when you ask for a time slot.
Yeah, my office hours, then also supply me already with your concrete question.
Tell me which problem you want to talk to me about, and where the problem actually is in the exercise that you have been working with or to plan the already a solution which you thought could be perhaps the right solution or we have doubts that it is the right solution
so that I can prepare for up with you appropriately.
So that would make the office hours more focused than they sometimes are with students.
And meet me unprepared and perhaps ourselves.
Okay, any other questions.
Just one more remark on how the lectures are given and usually of course I talk and talk and talk.
And please feel free to interrupt me anytime.
I mean, in particular, since this is digital format here and your questions will come in by chat. This is by no means and reasons to my lecture.
I will usually note that there is a question that has come in, and I will interrupt at a time, which is convenient to me and to the floor of my lecture to read out the question and answer the question.
So, please do not hesitate to ask me anything.
But it would be really helpful if when you pose a question that you pose it as precisely as possible in your chat.
Notice, because sometimes it happens to be the case that questions come in which I just don't understand. And since we don't have direct interaction due to the digital format I cannot really ask back and then the student responds and replies well
this or that, but rather the student has to sit down again and type again and so that costs time will usually or will perhaps lead to me progressing in the lecture because before the reply comes in.
So just really good if you take your time and write out a question as thoroughly as you possibly can.
Another question coming in whether the lectures will be recorded. Yes, indeed, they will be recorded in this lecture he is already recorded. I will typically upload the lectures at the latest one or two days after the lecture has given mostly I would try to do it on the same
day but sometimes the conversion of the recording, which is for some reason required by the university and done by the university, just takes time, and then I can't upload as fast as I would like to.
But, as I say, one or two days after the lecture the lecture should typically online, and I've given you the link in the organizational information to the lecture where you will find this.
This lecture recording it is not password protected can actually be accessed by anybody worldwide. And then you can really listen to the lecture in if you haven't understood the particular part.
Well, which is actually what I would recommend you to do in preparing for either the final exam or for any kind of problems or student contributions in between.
All right. Any other questions.
I don't see any more. I don't see anybody raising his or her hand. So let's start with the review of a probability, which basically follows Jeffrey
So this is of course also a source of information if you want to consult other sources that just my slides, even though I think that my slides are written rather carefully and extensively so that you can read them almost like textbook.
But this is of course my opinion, perhaps you differ from that opinion and find textbooks helpful nevertheless. So then one source, which would be applicable for this particular first chapter review of probability would be Jeffrey Woodridge's textbook.
In the fifth edition, I refer to here. Well, actually, it doesn't really matter which edition you use, you can typically also consult older editions than this one.
Okay. And before I actually start with systematic review of probability, I would just like to point out that probability theory is not just some abstract rather mathematical concept which we need for good science.
But that knowledge and probability theory can actually be really useful also in everybody's everyday life. And here is a motivational example for something which actually happened to a colleague of mine who happened to be a pediatrician and statistician and whose wife was pregnant.
And this woman was tested, a standard test for the mental health of the fetus at some stage of the pregnancy.
So that's a standard test which every woman undergoes. And in her case, it was the case that the test was positive, possibly indicating that the fetus was mentally ill.
So the woman was of course very concerned about this test result, and she talked to the medical staff at the institution where she underwent this test which took care of her.
Pregnancy and was told that test result is correct 90% of the cases.
And more or less she received the question, perhaps implicit I don't quite recall, would you like to have an abortion.
So that was deeply concerning to this woman that she went to her husband who as I said it was a professor of econometrics and said well, what does it actually mean when they tell you the test result is correct 90% of the cases.
Now, this guy read the literature, particular about the illness and the ways of testing for this mental illness, and what he learned was that in the women's age class, the one was already older than 30.
The prevalence of this illness is relatively high, namely one in 100, but subject to this type of mental illness.
So, the information they had at this stage was the test was positive.
The prevalence of the illness is relatively high and the test is 90% accurate.
So the question for them was, should the woman have an abortion.
Now, the husband convinced her not to have an abortion, and his reasoning was as follows.
Suppose that we randomly test 1000 pregnant women in the particular age class so aged older than 30 years.
We know the prevalence of the illness is 1%, so 10 of the 1000 pregnant women would have a fetus which has a severe mental illness, whereas 990 women would bear a healthy fetus.
Now suppose that the statement which the medical staff had given them correct 90% of the cases applies to all the fetuses.
Then this would mean that of the 10 women with an ill fetus, which has this mental illness, of the 10 women, 9 will have a positive test, and one woman will have a false negative test.
If the 990 women with a healthy fetus, there will be 99, which have falsely a positive test and 891 will have a negative test.
So 891 negative tests give the correct state of the illness or health of the fetus, but there will be 99 false positive tests if we assume that the test is correct in 90% of the cases.
So then 891 will be 90% of the 990 women here.
So in total, this would mean we would have 108 out of 1000 tests which are positive, which would indicate that the fetus may be or is a mentally ill.
Or in other words, invertive probability, the probability for a positive test result is 10.8%.
Now what we are interested in, or what this particular couple was interested in, is the probability of the fetus being ill, given that the test result is positive, because this is the information they have.
Test result is positive. So the question is, what is the probability that the fetus is being ill?
And the point I'm aiming at here is that the medical staff with his statement, the test is correct in 90% of the cases, seemed to indicate that the probability of the fetus being ill is 90%.
And this is wrong, as you will see now.
When we have 108 positive tests, as you recall, and we know in only nine cases of these 108 positive tests, the fetus is actually ill.
So the probability of the illness given a positive test is, well, I denoted here in this way, which you probably know from undergraduate studies, the probability of an ill fetus given a positive test is 9 out of 108.
Right? We have 108 positive tests, and there are nine fetuses of those women tested positively, which are actually ill.
So this probability is approximately 8%.
So this is what this colleague of mine explained to his wife. He told her, well, become the probability that our baby actually has this mental illness is not 90%, but it is 8%.
And so the woman decided not to have an abortion, and a couple of months later, she just gave birth to a perfectly healthy baby.
So the important point here is that with some training in probability theory, you would know from the data that the positive test result increases the risk of bearing an ill fetus, increases this risk from 1%, which every woman has in this particular age class, to 8%.
And that's a sizable increase in risk, but you have to know it is 8% now, given the positive test, it is not 90% as the hospital staff had implicitly suggested.
So this example is just an illustration of Bayes theorem, which you may know from elementary probability theory in your undergraduate studies.
The probability of an ill fetus, given the positive test, is equal to the probability of a positive test, given an ill fetus, times the probability that the fetus is ill, divided by the probability of a positive test, the unconditional probability of a positive test recorders.
So these three probabilities here we know, and from these three probabilities we can compute by Bayes theorem the probability of a fetus being ill when the test is actually positive.
We know the probability of a positive test result, given that the fetus is ill, is 90%.
We know the probability of an ill fetus is 1%, and we know the probability of a positive test result among all the women tested is 11%, 10.8%.
So, compute that, calculate that, this is 0.08, this is the conditional probability of the fetus being ill, conditional on the event that the test was positive.
And that's the whole point I wanted to make. Sub-training and probability theory can be really useful in real life and can actually be a matter of death and life for your baby.
Anyway, now we start with a systematic review of probability theory, and the first thing I want to go through are distribution functions.
So I will not bother to give you the exact mathematical foundation of what a random variable is, but rather start immediately with a description of the distribution of a random variable, where I for simplicity here first refer to discrete random variables and then a couple of slides later I come to continuous random variables.
So, the probability distribution for discrete random variable which we call x consists of a listing, listing of the possible values that the variable x can take.
And this listing is then matched with the probabilities of each of the values which variable x can take on.
So, for instance, if x can take on, let's say, k values, which we call small x1, x2, up to xk, then this is our list of possible values which the variable x can take, right? This is what I have written up here.
And then we can define the probabilities p1, p2, up to pk as probabilities taken from the close to about 0.1, such that we say, well, pj is the probability for our random variable x taking on value small xj.
That is the probability for event xj actually taking place, right? So, for each possible event of our list of events here, we can define what is the probability of this event.
And obviously, all the probabilities should sum to one, right? We will express probabilities almost always as decimal numbers, so not as percentages. Obviously, percentages should add up to 100%, probabilities decimal notation add up to one.
From this concept, we can just define the probability density function or PDF. The probability density function of a discrete random variable x is given by a function f, which maps the space of real numbers r on the closed interval 0, 1, such that f of xj,
f is the probability density function, right? This function here, f of xj is equal to the probability for event j for all the possible k events,
and such that f of x is equal to 0 for all x in the space of real numbers, which are not an element of the possible events on this random variable.
For instance, if you throw a dice, right, then there are six events which you can have numbers 1, 2, 3, 4, 5, 6, and you cannot roll a 2.5, for instance, or the number pi or something like this.
So, any x which is not the number 1, 2, 3, 4, 5, 6 would have a probability of 0.
This is why we can define the probability density function as a mapping from the space of real numbers to the interval 0, 1, because there would be many, many events which are just not possible,
and which would then be assigned a probability of 0, and only the possible events would get a positive probability.
Rolling a dice is one example. Tossing a coin would be a different example. So, assume that you toss a coin three times, and denote by random variable x the number of heads thrown after tossing a coin three times.
So, the coin has two possibilities, either heads or tails. You throw it three times or toss it three times. So, the events which may occur are different combinations of heads and tails, three different combinations of three different outcomes for tossing either head or tail.
So, one possibility would be that you toss t, t, t, so three times tail, which means that if x counts the number of heads, then you haven't tossed any heads, so x is equal to 0.
Because we have tossed three times tail, t, t, t, then it's easy to compute the probability of that. Obviously, the probability of tossing one, tossing t just once is one half.
The probability of tossing t twice, successive tosses, would be one quarter, one half times one half, and tossing t three times would be one eighth, so one half raised to the power of three.
The probability of x being equal to one is more complicated than that.
This means that one of the three tosses here has resulted in a head. So, it can be either the first toss which was the head and the other two being tails, or it can be the second toss which resulted in the head and the other two being tails,
or it can be the third toss which resulted in the head and the other two were tails. So, these are the three events which can happen under which our random variable x would take on value one.
So, obviously, the probability of each single event here is again exactly one eighth.
So, the probability of tossing head in the first toss is one half, times the probability of tossing tail in the second is one half again, and here another one half, so the product of the three is one eighth, plus one eighth here, plus one eighth here, gives three over eight here, three eighth.
The probability of tossing head two times, x equal to two, is made up of the same type of thought.
We would have two events of head and just one event of tail, so that's basically this thing here just reversed.
Either we have the two heads here, or we have the two heads as the two last tosses, or we have first and third toss as head.
So, basically, this is the same probability as tossing tail just once, either as the third toss, or as the third toss, or as the middle, the second toss.
So, the probability is again three eighth, and then of course the probability of x being equal to three is the probability of three heads, and that's of course one eighth.
If you add all those probabilities up, then you have one eighth, plus three eights, plus three eights, plus one eighth gives you eight eights, so this adds up to one.
And therefore, the probability density function of x for the simple coin tossing experiment is just f of x being equal to one eighth, 0.125 here, three eights here, three eights here, and one eighth here, and zero for any other x element r.
So, one eighth, 0.125 is the event that x takes on the small value x equal to zero, or 0.375 takes on, is the probability for the value small x equal to one, and so forth.
So, that would be the probability density function, and we can write this in the form of histogram, where we would say, well, here we have the probability density.
So, we know that tossing a point three times and having no head at all would mean x is equal to zero, so the zero is actually here, right, so this has a probability of 0.125, which you see here.
And any greater value doesn't give additional probability, so the probability of tossing something which is smaller, which is greater than zero, but smaller than one is of course zero, so the probability density function is flat here.
And then the PDF jumps to 0.375 here for the event of having just one head, and 0.375 for having two heads, and then 0.125 for having three heads out of three tasks.
Now, from the probability density function, we can easily construct the cumulative distribution function, which is abbreviated CVF.
The cumulative distribution function of a random variable x, and now not only a continuous, not only a discrete random variable, but general random variable, is given by a function capital F, which maps r into the close to them are between 0 and 1, such that f of x is the probability for our random variable capital X,
taking on a value which is smaller than or equal to some small x, some number small x.
So as I said, this definition here is applicable to all kinds of random variables, not only to the discrete random variables for which I have so far only abstracted the PDFs.
But I will come to the probability density function of a continuous random variable in a minute.
Okay, so this applies for all type of random variables, and now we move back to the case of the discrete random variable, then the cumulative distribution function capital F of x just sums the probabilities pj.
Namely, for all j, for which it is true that xj, the outcome, is less than or equal to the preassigned number x.
So f of x is the sum over the pj's multiplied by what is called an indicator function. This function here, one with an index of a set, I mean the set of numbers xj, which are smaller than x.
This function is one if xj is less than or equal to x, and it is zero if xj is greater than x.
So this would ensure that while the sum here goes over all events, we have k possible events for our discrete random variable,
we would ensure by this indicator function that all the probabilities which are associated with an event, which is greater than x, so sum of xj which is greater than x, all these probabilities would be multiplied by zero, so they would not be summed.
Just the probabilities which are less than or equal to x would be summed here. The probabilities of events which are less than or equal to x would be summed here, and therefore this expression is the CDF, the cumulative distribution function.
So when we return to the coin tossing experiment, we have the cumulative probabilities, f of zero is 0.125, so one eight.
Before it was, so f of some number x which is smaller than zero was of course zero, and now the probability for having no head, no three tosses at all, so for x being exactly zero, this probability is 0.125.
f of one then would be the sum of 0.125 and 0.375, so this is the probability for having either zero heads or one head among the three tosses of the coin.
And f of two then adds another 0.35 to that, so 0.875 would be the probability of having either one or zero or one or two heads among the three tosses.
And then of course, f of three is just equal to one because this would cover all possible outcomes of the experiment.
So the corresponding cumulative distribution function is then f of x is zero for x smaller than zero, it's 0.125 for x being greater than or equal to zero and smaller, strictly smaller than one.
0.5 for x being greater than or equal to one, but strictly smaller than two, 0.75 for x being greater than or equal to two, but strictly smaller than three, and from three onwards, the value of f of x is less than one.
So the cumulative distribution function looks like this, it is a step function, right? An increasing step function, not strictly monotonously increasing, but just monotonously increasing because there are parts of the step function,
well, the essence of a step function, where there's no increase in the value of the function.
Clearly, this function, as you see it, is discontinuous, right? It has jumps here and there and there, and of course an initial jump, which was here. It's not a continuous function.
So this was the question, how about continuous random variables?
For a continuous random variable, the probability of x hitting a specific number, small x, just some real number, is always zero.
It often comes as a surprise to students. So if the random variable is a continuous random variable, and I will now define what actually is a continuous random variable,
then the probability of hitting a specific x element r is exactly equal to zero.
Now, what exactly would be a continuous random variable, or how do we define it? The definition is very easy. A variable x is a continuous random variable if its CDF is continuous.
This CDF here was discontinuous. It had jumps. But if we have a smooth function as a CDF, then the random variable, the underlying random variable, is called a continuous random variable.
So if the CDF is continuous, then the random variable is continuous.
Now, a continuous random variable takes on any real value with a probability of zero.
So this seems counterintuitive at first, because you would think that if you add up the probabilities of all possible events, then you must arrive at one, like in the discrete case.
But for continuous random variables, we cannot just add up those events. Or if you think of it in these terms, then we would come to the apparent paradox that all events have a zero probability.
Now, why can we not add up just probabilities? Well, the simple reason is that here we have a continuum of outcomes.
And a sum is only defined for, well, not really finitely many, but countable many events.
It is not defined for continuously many events, so super-countably many events. For this case, we would need an integral.
An easier way to understand why the probability of hitting a specific small x taken from the space of real numbers, why this probability is zero, or why there is not really a well-defined probability for that, but rather identity, is given by the following proof.
So this proves now that the probability of any specific event is zero.
See, by the cumulative distribution function f of x, so the cumulative distribution function is defined as the probability of the event that x is less than or equal to a small x.
So that's a well-defined probability here, which covers an interval, basically, of possible outcomes of the random variable x.
In the same way, we can say that f of x minus some number epsilon, we think of that as a positive number here, a small positive number.
The value f of x minus small epsilon is defined as the probability that a random variable x takes on a value less than or equal to x minus epsilon.
This is the definition of f of x minus epsilon.
So from these two implications of the definition of the CDF, we can derive what the probability of x being exactly equal to small x is.
Not an inequality as here or there, but exactly equal.
The probability of x being exactly equal to x is obviously the limit of the difference between the probability that x is less than or equal to small x and the probability of x being less than or equal to x minus epsilon.
Well, these two things here are, however, equal to f of x minus f of x minus epsilon.
The limit of this difference f of x minus f of x minus epsilon is, since the distribution function, the CDF, is continuous, this limit is then zero.
So we have proven that the probability of x being equal to x is zero for any given x in the space of real numbers.
So that's general property of continuous random numbers. It doesn't make sense to speak of a positive probability that x is equal to small x if the random variable is continuous.
Now, in this setting, we have assumed that the CDF is continuous and we have defined continuous random variables by referring to continuous CDFs.
Obviously, we may also think about the question whether the CDF can be a differentiable variable.
You know that a differentiable variable is always continuous, a differentiable function, I should say, a differentiable function is always continuous, but a continuous function is not necessarily differentiable.
So suppose that the CDF f of x of random variable capital X is now not only continuous, but also differentiable.
Then first thing is we know that f of x is continuous, this is trivial, and thereby x is a continuous random variable.
But we can then also define the PDF, the probability density function, small f, as the first derivative of the CDF of the cumulative distribution function.
f of x is just the derivative with respect to x of capital F of x. Or equivalently, capital F of x is, if you integrate this equation here, the integral from negative infinity to x over f of t dt.
As you probably know from an aggregate statistics, if a and b are some constants with a being less than b, the probability that a continuous random variable lies between a and b is just the probability of a being smaller than or equal to x being smaller than or equal to b.
And that's then just f of b minus f of a. So this means the area under the probability density function f of x is, so between the points a and b, is just the probability of x being between a and b.
You see this here in this graph, which I've copied from the Woodridge textbook, we have two numbers a and b. This is the PDF, the probability density function.
And this axis here gives certain numbers x, which may be possible outcomes of random variable capital X, then we know the probability that capital X will be somewhere between a and b, a and b included, let's just play a role, is exactly equal to this area here under the probability density function.
So it's exactly equal to the integral of the probability density function between point a and point b. And we could have written it this way here, but you know this certainly f of b minus f of a is the same thing as the integral from a to b over f of t dt.
All right, any questions so far?
Raise your hand if there is a question, otherwise I will continue.
Okay, important properties of cumulative distribution functions are the following. For any number small c, it is true that the probability of x being greater than c is the same thing as 1 minus the probability of x being less than or equal to c, so it's equal to 1 minus f of c.
Moreover, for any numbers a less than or equal to b, it is true that the probability of x being between a and b, in the sense of strict inequality here and weak inequality there, is equal to f of b minus f of a.
And third, for continuous random variables, it is true that since the probability of any particular value x is zero, we have that the probability of x being greater than or equal to c is equal to the probability of x being greater than c.
Or the same thing for less than or equal to versus strictly less.
This, of course, here is only true for continuous random variables because their probability of hitting one particular value is just zero.
So the probability of hitting exactly c is zero, as we have seen. So this set of events described here is not in any probabilistic sense greater than the set of events here because the probability that x hits exactly c is equal to zero.
So which is different between this sign here and that sign there.
In the case of a discrete random variable, of course, these two things are not equal. It may well be that there's a positive probability for x hitting exactly c.
And then in this case, this probability here would be greater than this probability on the right hand side of this equation.
With these preliminaries, I would now come to joint PDFs and marginal PDFs. And again, I first cover the discrete case.
So what is a joint probability distribution? This is actually something we always deal with in econometrics because we typically just don't look at one event.
But we look at many events along many different dimensions which happen at the same time and which perhaps have some relationship to each other.
So joint probability functions are core to econometrics.
A joint probability distribution for now just two possible dimensions in which events can occur. So for just two discrete random variables which are now called x and y consists first of a listing of all the possible combinations of values that the two variables can take.
And second, the joint probabilities associated with each such combination. So for instance, when we have variable x which can have the outcomes one, two, and three, then the listing which we had for the one dimensional PDF would list three events, namely one, two, and three.
Suppose that variable y also has just three events which we call a, b, and c.
The listing of all the possible combinations of values would mean that we make a listing which starts with 1a, 1b, 1c.
Then 2a, 2b, 2c. And then 3a, 3b, and 3c. These are the nine combinations which are not possible. So from a list of three events for x and three events for y, we get a list of events of nine events, namely those nine events which I have just said. So three times three events.
This would be for the case where we do not pay respect to a sequence of events, right? If it matters whether we have first one and then a or first a and then one, of course, things are different and these would be different events and there would be even more outcomes, more events, more combinations.
But currently here we just draw simultaneously from x and y and compare which number we have drawn and which letter we have drawn.
So there would be a listing of all the possible combinations of values that the two variables can take on. And second, there would be joint probabilities associated with each such combination.
So for the nine events which I have just described, we would need nine probabilities and these nine probabilities, of course, should add up to one. So we have three possibilities for the events
for variable x and three possibilities for the events for y. Actually, we basically have just two probabilities here and two probabilities there because if we had three events, the third probability is always one minus the other two probabilities. So just two independent probabilities.
But now we have then nine probabilities or if you want to take the analog of the argument I just presented, eight probabilities and the ninth probability is one minus the sum of the former eight probabilities.
So, proceeding a complete analogy to what we have already done in the one-dimensional case, we denote a joint probability density function, joint PDF, for two discrete random variables as f x y of certain small values given real numbers, small x and small y, as the probability that
variable x, random variable x takes on value small x and random variable capital Y takes on small y. That would be the possibility for certain event in the xy space and there is a certain numeric value associated with this probability.
As joint probability density function, which describes the joint likelihood of the combination of events, we would then write small f indexed x y, capital X and capital Y at some point x and y, which would measure the event that capital X is equal to small x and capital Y is equal to small y.
Of course, the sum of all those probabilities in the joint distribution is then again equal to one.
So given this definition of a joint PDF, we may also define what the marginal PDF is.
If x and y are jointly distributed, then the probability density function f just an index x or f just index y rather than f index x and y as I have it here for the joint PDF.
So the probability density function f of x would be defined as the probability that x takes on value small x, regardless of what y is doing.
And similarly, f y of y would be defined as the probability of Y, capital Y being equal to small y, regardless of which value x is taking on.
So these two things are the marginal probabilities, probability density functions, and they are actually the same thing as the PDF as we had already discussed it or defined it for the one dimensional case because in the marginal distributions, we just disregard the other component.
Here, we disregard the y. Here, we disregard the x. So these are called marginal probability density functions.
Note again that these definitions do not distinguish between discrete and continuous random variables.
So in the discrete case, we would have that the marginal density function f x of x, capital X of small x, is equal to an infinite sum running from negative infinity to infinity over f x, y, well, at the point small x, and some point of y, j.
And we just here go over all possible realizations of y by letting the j run over all possible events, negative infinity to positive infinity, and then we sum all these probabilities, these values of the f x, y function to derive the marginal probability density function.
And in the continuous case, this is completely analogous, f of x of x, so the marginal density function of x, is equal to the integral for negative infinity to infinity over f x, y, at the point x and y, dy.
So the marginal PDF f of x, or f x of x, is just the probability of x being equal to small f, regardless of which value y assumes.
From those density functions or distribution functions, we can also derive the conditional distributions.
So we may be interested in the probability of y being equal to some small y, given that x takes on a specific value x equal to x.
Remember my introductory example with the positive test of the fetus, there we wanted to know the probability of the fetus being ill, so small y is ill, given that the test x
had taken on a specific value small x, in this case had been positive.
Well, the definition is very easy. The conditional probability density function is defined as a function y, given x, which is f x, y, so the conditional probability density function at point x, y, divided by the marginal probability of x.
Obviously, we have to ensure that this marginal probability is then different from zero at all points x. So we must ensure that f x of x is greater than zero.
And if it were zero, then clearly the conditional probability density function would not be defined.
From this, we see immediately that the joint density function is equal to the conditional density function times the marginal density of the conditioning variable.
So the way you have to remember that if you haven't already remembered the undergraduate studies is that conditional distribution function, f x, y, at point x, y, is equal to the
conditional probability function, the joint probability function here is equal to the conditional probability function, y, given x, so that's the condition that x is already given, that we already know that a particular value small x has been taken on by the random variable capital X, times the marginal distribution function f x of x.
And the same holds, of course, for y as the conditional variable or as the conditioning variable, f x given y is also a conditional distribution function, a conditional PDF at point x, given y, times then f y of y.
It was PDF and not a CDF, but I think it's there. Now in the case of a discrete random variable, we have that probability of y being equal to y, given the probability of x being equal to x, is equal to f of y given x.
Well, at a specific point, namely y, given x.
This is precisely the type of question which we typically ask in economic analysis. Namely, we ask, what is the effect of x on y?
Given that x assumes value small x, we would like to know what is the effect of this value of x on capital Y.
How likely is it then that capital Y will take on the value small y?
This is what this probability tells us, or what this PDF here tells us, this conditional PDF tells us.
So when we deal with econometrics, and when we want to talk about at least correlations and sometimes also about causality, we have to be aware that these type of distributions, conditional distributions of variables we are interested in,
on other variables which we may be interested in. Typically, we are interested in these variables, which we may observe, but sometimes we don't observe.
We want to know what the influence of these variables on the y variable is.
Therefore, the conditional PDF tells us how a change in x affects the likelihood of observing y.
Now, if x and y are independent random variables, and I haven't defined independent random variables yet, but I will do this later,
and probably know what independent random variables are, then the knowledge of the value of x tells us nothing about the probability that y takes on various values, and vice versa.
So if it is true that x is independent, then we will see that the value which is taken on by variable y does not depend on variable x, so it has no influence on it.
In this case, therefore, we have that the conditional PDF here is equal to the marginal PDF, which we have here, fy given x is the same as fy.
And the same thing, of course, holds here for f of x given y. So again, the conditional PDF here is equal to the marginal f here when the two variables x and y are independent.
Equivalently, and please try at home to verify that the joint distribution, the joint PDF of xy, is equal to the product of the two marginal probability density functions fx and fy.
Yeah, this was the first part of chapter one. I think I leave it at this for today. We still have one or two minutes. Are there any questions concerning the stuff which I have done?
Please raise your hand if you do have questions.
If there are no questions, then perhaps could you tell me via chat, was this all boring because you knew it already?
Was it helpful that I went through it again, even though you had covered it already, or haven't you heard it at all in your undergraduate studies, just to give me some idea on what your level of education is?
I will stop the recording already, and perhaps some of you type in some of the answers. Yes, somebody tells me very helpful as a refresher. Thank you very much. Any other reactions?
Great and necessary refresher. Thanks.
Good revision. Okay, so this seems to be all in the same line of reaction. Yes, very good. Anybody who found that rather boring and unnecessary?
Okay.
Somebody writes, I've learned this at undergrad program that thanks for reviewing it anyway.
Okay, so this seems to indicate that it was not all wrong to go through this thing and I will continue to do this next week. Thanks for the reactions so far.
Not next week, tomorrow, of course. And so we'll see or hear each other again tomorrow, and then I will continue the review of probability. Enjoy the rest of your day.