Thank you.
Good morning, everybody, and welcome to Time Series Econmetrics. Are there any questions? Then please raise your hand.
If you don't see any questions, let me just remind you that what we did in the last lecture was that we discussed certain concepts like stationarity and ergodicity of a time series.
And the important point you should bear in mind is that stationarity and ergodicity are different though related concepts, and perhaps an easy way to remember what one concept or the other concept is about is to make it clear to yourself that stationarity is something which implies something about the theoretical
properties of a time series in the sense that stationarity implies that the mean and the variance and the covariancies of a time series are constant over time.
Whereas ergodicity is a concept which implies something about the empirical estimates of moments.
So stationarity says something about the theoretical moments we want to estimate, the concepts which we need to have an objective in our estimation, namely mean, variance and covariancies.
Whereas ergodicity says something about the estimates themselves, namely about properties of the estimates, since ergodicity implies that it is possible to estimate the theoretical moments consistently.
So this was essentially what we discussed in various aspects in the last lecture. And today I would like to move on and talk about univariate stationary processes.
The point of departure here is the famous book by Box and Jenkins from 1970 time series analysis forecasting and control, which is more or less the cornerstone of modern time series analysis.
Because what Box and Jenkins did was that they developed a complete and comprehensive set of tools to analyze a certain class of time series models, essentially all linear stationary time series models.
And these linear time series models and as you would see also the linear stationary or in particular the stationary linear models are divided into three different classes of models, one of which is called the autoregressive models or AR models.
The other class is called the moving average models or MA models. And then of course you can combine two models and then you get what is called an ARMA model.
And these type of models are used everywhere in time series analysis. We later in this lecture expand the Box and Jenkins framework in the sense that we also look at in stationary time series, so time series where the mean and the variance is not necessarily a concert or is not a concert at all.
But currently we just stay with stationary processes and also we just look at univariate processes, multivariate processes are coming later in the lecture.
So we're just looking at one variable at a time, at a point in time, but we look at different points in time, but always just one single variable whereas later we will look at vectors, which comprise different variables measured at one point in time and through
time, which would then be multivariate time series analysis.
We start with the simplest possible time series process of this class of autoregressive processes.
But before we go to the simplest case, namely the AR1 case, let me give you a general definition of what an autoregressive process says is, you see the definition here.
Let's UT just be a process of white noise, so something which is completely unstructured, just random events distributed everywhere over time. So in each period, we have some shock, some shock which is uncorrelated with all the shocks everywhere else in time.
And then we have a set of real numbers, nu and alphas, alpha1 up to alphaP. Let's say alphaP is different from zero. If we have these numbers as given, then we look at the expression highlighted here in yellow, expression 13, which is a stochastic difference equation, namely that
yt is nu, so constant term, plus alpha1 yt minus 1 plus alpha2 yt minus 2 plus plus plus plus plus, so it goes on like this, plus alphaT yt minus P plus some shock.
So essentially equation 13 says we can derive yt by looking at the past development of yt, yt minus 1, yt minus 2, up to yt minus P if it is an ARP process.
So yt turns out to be a linear combination of these P values the process has taken in the past, plus a constant nu plus some noise, ut, some error term which just materializes in period t, so which did not affect the values yt minus 1, yt minus 2, yt minus P and so forth, because ut occurred only in period t, to put it intuitive.
So this stochastic difference equation is set to describe a univariate time series, or it describes a univariate time series yp, and we call it an autoregressive process of order P or ARP.
So this is why I have here said that alphaP is different from 0, because clearly if alphaP were equal to 0, then the process would be at most ARP minus 1, because clearly the coefficient which is equal to 0 would cause the yt minus P to drop out from this equation here.
So essentially we want that the highest coefficient which we have here, highest referred to the highest lag in this equation is nonzero.
Some of the other alphas here or even all of the other alphas may be 0, but if the alphaP is different from 0, then we call this an autoregressive process of order P.
And here we look at the simplest example, as I have already announced, which is an AR1 process, so we just have one lagged value of y here, which is the previous period's value of the y process.
This coefficient, according to our definition on the previous page, would be alpha1, but I dropped the subscript 1 here since we just have one alpha coefficient and just denote the process as yt is equal to constant nu plus alpha times yt minus 1 plus wet noise ut.
So that's the easy case of an AR1 process, and we will first analyze this process before moving on to the slightly more complicated case of an AR2 and then an ARP process in general.
Now one important thing you have to bear in mind when dealing with time series is that this equation here is valid for all periods t,
which is why I actually called this here a stochastic difference equation because this also means that this equation here holds for all periods t, not just for one single period in which this is true, but in each and every period, this relationship here holds true.
So in particular, if we know that yt is equal to nu plus alpha wet t minus 1 plus ut, then we know that we do have the same relationship in period t minus 1, so yt minus 1 is equal to nu plus alpha yt minus 2 plus ut minus 1.
So up to here, this looks like a really easy linear process. All the mathematical operators here are just operators which are linear operators.
We have the same thing here, but now using this definition here in equation 14, substituting of the yt minus 1 here by the expression on the right hand side of the second equation here, we get yt is equal to constant nu plus alpha times this value, which is yt minus 1 plus ut.
So that we can sort out terms and we now see that the constant changes. The constant is now nu plus alpha times nu, so we have as a new constant 1 plus alpha times nu plus alpha squared yt minus 2 plus ut plus alpha ut minus 1.
If we sort out terms, then we have now a new constant of this form here. We see that yt is not explained by yt minus 1 anymore, but it is explained by yt minus 2 and it is explained by a new error term.
Now this may perhaps confuse you because you may think, well, suddenly the ar1 is an ar2 because now yt is explained by yt minus 2 and apparently the coefficient for this yt minus 2 term is alpha squared, so it is different from 0, provided that alpha was different from 0, which of course I have here assumed because otherwise it wouldn't be an ar1 process.
So it seems that we have transformed the ar1 process into an ar2 process and you may ask is the distinction between ar1 and ar2 and more generally between other forms of ar processes actually a sound and reasonable distinction.
And yes, indeed it is because recall that in the definition of an ar process we required that what we add to constant and lag terms of the y was an error process which is white noise, just ut.
But here we don't do this anymore. Here we add ut plus alpha ut minus 1, so these two terms together are not white noise anymore.
Very clearly they are correlated over time because ut correlates, or let's say if we call this term here vt, like a new, give it a new name and call it vt, then this vt would correlate with vt minus 1 because we have here ut and ut minus 1 in here.
So the sum of these two is not white noise anymore and therefore this process here is not an ar2 process, rather it is just a transformed ar1 process.
Now, as you may already have expected, this process we can continue. So currently we explained the yt by yt minus 2.
But as I have said, we can shift equation 14 any direction in time and any number of periods we want to. So here we have shifted it just one period backwards, we can also shift equation 14 two periods backward or this second equation here another period backward, make it yt minus 2 is equal to nu plus alpha yt minus 3 plus ut minus 2.
So that would also be a valid equation. We can then substitute of the yt minus 2 here and substitute in an expression which involves yt minus 3.
And we can do this many, many times. If we do this tau times, tau substitutions, we get the general formula that yt is equal to nu times the sum from index i going from 0 to tau over alpha to the power of i.
Because this is what begins here already, this was alpha to the power of 0 plus alpha to the power of 1. We do this one more time, then we would have a new term in this parentheses here plus alpha squared, so alpha to the power of 2.
And after tau substitutions, we would have nu times the sum here.
Then apparently here we have alpha squared. First we have just alpha to the power of 1 times leg y, then we have alpha to the power of 2 times yt minus 2. So the exponent here corresponds with the leg, which we have in here.
This case leg of 2 period, this case leg of 1 period, and this would also be the case here. So we would have alpha to the power of tau plus 1. So one unit more than the upper limit of the sum here times yt minus tau minus 1. So minus parentheses tau plus 1.
Same thing as this exponent here.
Plus then the term alpha to the power of i, ut minus i, which adds up white noise over tau plus 1, two periods from 0 to tau. Sorry, but I think no, this was wrong.
Here it was, right? So this equation here is true for any tau equal to 1 or 2. So equal to 2, we have already, no sorry equal to 1, we have already in this equation here.
Equal to 2, you can easily compute at home. So you can verify this expression here, equation 15.
Now if alpha is an absolute value smaller than 1, this expression converges for tau going to infinity to an expression as in 16. yt is equal to nu over 1 minus alpha plus an infinite weighted sum of white noise terms, which shift over time.
So in this case, alpha to the power of i times ut minus i. So this representation here is still an AR1 process just written in a different way.
And this way should look familiar to you, or I hope it will look familiar to you after I've explained it, because equation 16 here is just the void decomposition, which we had already set up in equation 7.
So the void decomposition for yt, where yt is written as some deterministic term, in this case, nu over 1 minus alpha, plus then an infinite sum of white noise terms, weighted by something which in the void decomposition was called psi i.
And here the psi i is equal to alpha to the power of i.
Well, as well become a little clearer later, you will see that this representation here is the void decomposition only in the case where alpha in absolute value is smaller than 1, because actually only then this infinite sum here converges.
But we will see this very easily in a minute.
Perhaps you can do parts of this exercise already as an exercise at home and think about the question why equation 16 is not a void decomposition in the case of alpha in absolute value greater than or equal to 1.
Now it is a little cumbersome to go through these calculations and do all those substitutions, which I have done here, set up these sums here and then see to which kind of sum all of this converges as in equation 16, an easier way would actually to use the lag operator.
And that's why I have introduced the concept of the lag operator two weeks ago.
And so let's use this concept now and rewrite the AR process, the AR1 process, as 1 minus alpha L times yt is equal to nu plus ut.
So what I have done here is that I have moved to the left hand side of the equation, the term alpha times yt minus 1.
And since yt minus 1 is simply L times yt, using the definition of the lag operator, we can write this as alpha L yt and then we can factor out the yt and have 1 minus alpha L times yt is equal to nu plus ut.
We will suppose again that alpha in absolute value is smaller than 1.
Now we know that an inverse expression for this lag operator expression exists that we have shown already in equation 5.
So now let us multiply this whole equation here just by the inverse of the lag operator to get yt is equal to 1 over 1 minus alpha L times in parentheses constant plus 1 minus error term ut.
Right. So, this is the inverse lag operator and we know this inverse lag operator is actually the result of a convergent geometric sum.
This is true as long as alpha is smaller than 1 in absolute value. So we can write 1 over 1 minus alpha L as an infinite sum, which goes from 0 to infinity of alpha to the power of i L to the power of i times, same thing as up here, constant plus 1 noise error.
Now this means we have to apply the lag operator with the power of i to nu and to ut.
Applying this to nu doesn't change anything because nu is constant over time. So the lag operator is just ineffective and drops out.
Applying the L to the power of i to ut means that we have to shift ut by i periods backward in time if the lag operator is L to the power of i.
So we get, if we multiply this out here first, just split up the two terms, two infinite sums with alpha to the power of, sorry what happens here, alpha to the power of i L to the power of i times nu, which as I say is just nu because nu is constant over time.
And here we get the same expression with L to the power of i times ut, which is the same thing as ut minus i as in this expression here.
Now this thing here involves again the geometric sum because the nu can be factored out. So we have nu over 1 minus alpha, that's the new constant, plus, and then here the same sum as in the equation before, plus the sum over alpha to the power of i ut minus i.
And this is of course exactly the same result that we had already derived here in equation 16. I'm just now using lag operator notation and mathematically we arrive at the same result, not surprising.
Why is this representation helpful? At first it looks like it is much more complicated than the easy representation of the AR process we started out with, so this representation of the AR1 seems to be much easier because we don't have any infinite sum in there.
But from this equation here it is perhaps a little harder to compute what the expectation of yt is because if you take the expectations operator and put it over this equation here you would have expectation of yt is equal to nu, well expectation of nu is equal to nu, right?
So expectation of yt is equal to nu plus alpha times the expectation of yt minus 1 plus 0. But since we don't know what the expectation of yt minus 1 is, we also don't know what the expectation of yt is.
So this doesn't really help us by much and therefore it is very useful to use this representation here because if I now take the expectations operator and put it over both sides, then I would get the expectation of yt is, well this term here, this is a constant, so it's equal to its expectation.
So it is nu over 1 minus alpha plus, well infinite sum over alpha i times the expectation of ut minus i. But since ut minus i is white noise we know that the expectation of white noise is always 0.
So this whole term here vanishes because all the expectations of ut minus i are equal to 0.
So we are just left with nu over 1 minus alpha and now know that the expectation of yt is nu divided by 1 minus alpha and I define this as a new parameter mu. Mu is now the expectation of this process.
And the important property we immediately see from this simple equation here is that the expectation of yt is independent of t.
So all yt's have exactly the same expectation, the mean is constant over time. So we have mean stationarity for the AR1 process.
And we will see in a minute that we also have covariance stationarity or variance stationarity. But here's the first step for proving that an AR1 process with a property that alpha is smaller than 1, absolute value, that this process is stationary or in this case we have so far just shown that is mean stationary.
Okay.
Now look at the variance of yt. From the volatility composition we know that v of yt for any process which has a volatility composition is the same thing as, of course, by definition,
the expected value of yt minus its mean minus mu squared. And obviously if the process has a volatility decomposition, this is equal to sigma squared u times the sum over the psi i squared.
Right. Because we know from the volatility decomposition that yt minus mu is equal to this term here.
So equal to this sum here times psi i ut minus i. And since all the ut's are white noise, the squares of this sum here and taking the expected values of these is just the sum, the infinite sum over the psi i squared. This property we have already proven in, I think, the previous lecture.
Now, so we know this already from our analysis of volatility composition. We just have to apply this knowledge now to the new framework of our AR1 process in which the psi i coefficient happened to be alpha to the power of i.
And that was true for all i with alpha being smaller than one absolute value. So the variance of the AR1 process is then given as, well, variance is gamma of zero.
That's just our standard notation. Gamma of zero is equal to the variance of yt and the variance of yt is equal to the expression which I derived up here, which I
recall it for you up here. It's sigma squared u times the infinite sum of the psi i squares. Well, psi i is alpha to the power of i, so I can write alpha to the power of i squared, which gives alpha to the power of 2i.
Well, then this here is, of course, just a geometric sum again where alpha squared is taken to the power of i. So we get this that this is equal to sigma squared u divided by 1 minus alpha squared.
And this thing I call sigma squared y.
I had already sigma squared u here, which was the variance of u. And now I know what the variance of y is. It is this expression sigma squared u divided by 1 minus alpha squared.
This variance, as you see, is independent of time, at least in the case where alpha is smaller than one absolute value.
And you already see where we need this assumption that alpha is smaller than one absolute value because if alpha were equal to one absolute value, well, then we would not be in a position to divide by 1 minus alpha squared because obviously we are not allowed to divide by zero.
So we know that not only the mean but also the variance of an AR1 process is independent of time for alpha being smaller than one absolute value, and we will make this assumption in the following often.
Sometimes mostly I will state it again explicitly, but sometimes perhaps I will not do so and then you will just, unless I have said anything differently, you will just follow this assumption that alpha is smaller than one absolute value.
We have now shown here in the lecture that we have mean stationarity and that we have variance stationarity. As an exercise, I'll let you prove that there's also covariance stationarity.
So show at home, please, that the covariance is gamma or lag t minus s are also independent of time, which is to show that they just depend on the time difference tau, which we said to be the difference between t and s.
So assume that tau is greater than zero without loss of general reality because, as we know, the covariance function is symmetric so it doesn't depend on whether tau is positive or negative.
And then please verify that gamma of tau is the same thing as alpha to the power of tau times gamma of zero,
which, as we know, is then alpha to the power of tau times sigma square y because we have computed this already.
If you have proven that or when you have proven that this immediately implies that gamma of tau is equal to alpha times gamma of tau minus one, and that holds for all time lengths tau.
So we know that the auto covariances are also autoregressive.
Not only the time series is autoregressive, so not only the observations we have are autoregressive with just one lagged term, or in this case, AR1.
But, as you see, the auto covariances are also autoregressive because gamma of tau depends with coefficient alpha on gamma of tau minus one.
So the property of an autoregressive structure is transferred or holds also for the auto covariances and not just for the observations, which I think is an important and interesting insight.
In some sense, equation 19 here is easier than the original description of the AR1 process because here we have a homogenous first order difference equation, so an homogenous linear first order difference equation.
The emphasis is on homogenous because now we do not add a constant or some error term, but we have a very, very easy description for the behavior of the auto covariances of an AR1 process, mainly that the auto covariances fall geometrically.
With coefficient alpha, well, alpha is smaller than one, so the auto covariances become smaller over time, the higher the lag tau is.
We can also express this in terms of the auto correlation function, which a different word for that is the common logarithm of the AR1.
You see that the auto correlation function rho of tau is then just equal to alpha to the power of tau, which again says that we have a geometrically decaying auto correlation structure in an AR1 process.
Geometrically decaying means all the auto correlations of the AR1 are different from zero, because alpha to the power of tau is always different from zero, but an absolute value these auto correlations form, so they approach zero asymptotically
since alpha is smaller than one, in absolute value.
If I take this idea or illustrate this idea, this concept with data, look at the time series X1, which I have supplied in my data set artificial data, which you find on Steenen, and look at the auto correlations which are estimated.
These are true auto correlations because for time series X1, you just have the data, you don't know which time series process actually generated this data.
But what you can see here is that the auto correlations which are estimated become smaller over time, but stay at least for the first four observations until here, they stay positive.
Here are the estimated values 0.57, 0.30, 0.10, and then 0.009.
Then we observe a number of very small or rather small negative values, which should not be the case in the case of an AR1 process.
So this may cast some doubt on the question of whether this is indeed an AR1 process, but you also see from this graph here that these negative values are probably not significantly different from zero,
because they lie here in the confidence interval, which we had constructed last lecture. So it may well be that these auto correlations are actually zero or are even positive with a very small value.
And for some reason, we estimate something which is negative here. There's actually a systematic reason for why these estimates and finite samples are always a little lower than they are in reality, but we will come to this issue a little later.
So we may actually ask the question by looking at what we have estimated here correctly for time series X1 in this set of artificial data. We may ask the question, well,
can it be that the true process which generated the data in X1 is an AR1 process? That you do not know, and I won't tell you.
But from the correlogram, you might suspect that perhaps the true process is an AR1 process. So let's make a little empirical check. It's not really a fair, good test, but just to give you some idea on how you can
derive easily at certain hypotheses. Suppose, for instance, the time series X1 in this data set is in fact an AR1 so that there was an AR1 process at the root of this artificial data which I generated.
And suppose that the estimate of the first autocorrelation, which I denote by rho hat of one, is coincidentally the true value of alpha. So alpha would be 0.572.
So what I do now is that I say, well, let's take this estimate here, 0.572, literally, and assume that this is the true value of alpha in an AR1 process which generated the data.
So what would we expect to be, or what should be, the true second order autocorrelation if the first order autocorrelation is truly 0.572? And that's the question.
And the answer to this question is here in equation 20, the first order autocorrelation is alpha to the power of one.
The second order autocorrelation for tau equal to two would be alpha to the power of two.
So we would need to square the value alpha, which we had here assumed to be equal to the estimate 572. And if we do this, then we would say, in case that the true alpha is 0.572, then we know that the true autocorrelation at lag two is 0.572 squared, which is equal to 0.327.
We have, however, estimated at lag two, 0.301, which is perhaps not so far apart from the true value, which would be the true value if indeed this is an AR1 process and alpha is equal to 0.572, which we don't know.
But at least at first sight on an intuitive basis, we might say that the 0.301 is reasonably close to the moment which we would expect if the hypothesis is true that this is AR1 with coefficient 0.572.
Okay, so here I write this seems to be reasonably close.
Now let's look at the third autocorrelation. The third true autocorrelation would be rho of three, which is 0.572 to the power of three, which is 0.187.
What we have estimated is 0.104. And here I would say that 0.187 is not so close anymore to 0.104, even though the distance is approximately the same or even lower actually, actually much lower than the difference here.
But in percentage terms, I mean, this is almost twice as much as what we have estimated. So perhaps that's not such a good estimate or perhaps this is not really an AR1 process with coefficient 0.572.
So I would say that's not so close what we have found here. We should have found something in the order of 0.2 and we have found just something in the order of 0.1.
How is it for the fourth order autocorrelation? Well, the true value if alpha is really equal to 0.572 would be 0.572 to the power of four, which is 0.107.
We have estimated, however, only rho hat of four to be 0.009. So this again is not so very close. It actually differs from the moment that we would expect by a factor of 10.
So we cannot be really sure that the data was generated by an AR1. And this is why I draw the conclusion that our evidence does neither really reject nor confirm the hypothesis that this is an AR1 process with alpha equal to 0.57 true.
For this hypothesis speaks the fact that the first four autocorrelations fall monotonically as they should if the process were indeed an AR1 process.
Although it is true and speaks for this hypothesis that rho hat of two looks pretty good, provided that it is true that alpha is equal to 0.572.
But rho hat of three and rho hat of four are less convincing if alpha is equal to rho hat of one.
You may object now and say, well, we don't really know whether it is true that alpha is equal to the estimated value for the first autocorrelation. So it may not be the case that alpha is equal to rho hat of one.
Suppose, for instance, that rho hat of one was a little larger than the true alpha and the true alpha is equal to 0.53.
Well, in this case, the estimated and the true autocorrelations would be much better in line with each other.
For instance, if alpha were 0.53 rather than 0.572, we would have rho of one is equal to 0.53. Well, this is what we have estimated, 0.572.
So that's rather close.
For rho of two, we would expect a value of 0.281.
Here it's 0.301, which is also rather close.
For rho of three, we would have estimated 0.149, so 0.5.
Essentially, true value is 0.1. It's a little lower, but not as dramatic as before.
And for the fourth autocorrelation, we would have 0.008 as here, and here it would actually be 0.009, which is still quite a bit apart from the value that we have estimated.
So while the fit is perhaps better with alpha equal to 0.53, it's still a little inconclusive what we have here in terms of empirical and theoretical autocorrelations.
And the question of whether they do match or don't match may perhaps just leave left to be open.
The best thing would be that we have additional instruments beyond mere autocorrelations, and we will develop such instruments later, because then we develop the concept of a partial correlation, which is the second graph here, which I haven't talked about at all so far,
which you cannot understand so far. But we will come to this, and this will help us a little bit more to find out whether this is, in fact, an AR1 process and will support certain conclusions we draw about the time series properties of the series we are studying.
All right, exercise for you. Estimate the first four autocorrelations of X2 in the same data set.
And compare the estimates with the theoretical autocorrelations of an AR1 process with alpha equal to rho hat of 1.
The question is, what conclusions can you draw from this analysis? So just do the same thing which I have just done with you for a different time series in this time series set artificial data.wf1, which is an eViews data set.
Now I would like to illustrate to you that this look we have had at our autocorrelation structures can be quite helpful to distinguish between series, which at first sight might look rather similar.
So here I take again the artificial data from this data set, which you find on Steemit.
But now I look at series X1, which we have already analyzed, where we have computed the color logarm, and at series X5 where we haven't computed the color logarm so far.
And first I just look at the two series in the time domain. So here they are just plotted as time series, the X1 series is blue, the X5 series is brown.
I have used here the same white noise shocks to generate X1 and X5. So I saved the white noise shocks and then used them to construct the series X1 and X5.
And I think you will agree when I say it seems that X1 and X5 are very, very similar in terms of, well, time domain properties.
There's some deviations, for instance, here between blue and brown. Some times here or there. But by and large, it seems to be the case that these series, well, almost coincides.
Now let's look at the autocorrelations of X1 and X5, the estimated autocorrelations, I have to say, because we don't know what the true autocorrelations are.
And here it turns out that the estimated autocorrelations of X1 are quite different from the autocorrelations of X5.
See, for X1, we have this property that, well, the autocorrelations decay and perhaps they decay geometrically as they should in the case of an AR1 process.
For X5, there's certainly not a geometric decay in autocorrelations. Rather, we have a rather high first-order autocorrelation.
And then we just have things which are insignificant, positive here, negative already here, then going on to be negative.
So the basic and most important autocorrelation is actually this X5, this first leg here of variable X5, and it's not so clear whether anything significant happens after a period, after lane one.
So we can be very sure, actually, that process X5 is not an AR1 process because the autocorrelations structure here is so different from what we should see for an AR1 process as the AR1 has geometrically decaying autocorrelations.
That may be the case for X1, a little inconclusive, as I have argued, but it is certainly not the case for X5. So I think we can safely exclude the possibility that X5 was generated by an AR1 process.
And this is, in fact, true X5, certainly not an AR1 process.
Or I know that because I've generated the data using some type series model.
Okay.
The analysis we've done so far is an analysis for alpha smaller than one in absolute value.
I now come to a proposition for the AR1 with a coefficient alpha bar being either equal to one in absolute value or even greater than one in absolute value.
But before I explain to you this proposition, and it's proved, let me ask you, are there any questions for the case alpha smaller than one absolute value?
If you have questions, then please raise your hand.
Apparently not. Then let's go to the proposition, which says the AR1 is not stationary for alpha bar greater than or equal to one.
And the proof is indirect for this proposition.
What I do is that I first exploit the fact that a stationary time series, Yt, must have a constant variance.
So sigma square Y must be greater than zero, and of course must be constant over time.
Now assume that we have an AR1 which is stationary.
So assume that we have, for instance, an AR1 without a mean, so without a new term, so new is equal to zero for simplicity, so that we have Yt is equal to alpha Yt minus one plus ut.
What I do here is that I square both sides and take expectations.
So the square of Yt is Yt squared, and I take the expectation of that.
And this thing here I can also square and take the expectations of that, then I get alpha squared Yt minus one squared plus two alpha Yt minus one ut.
So the combined term here plus ut squared.
Oops, sorry.
Well, we know the expectation of Yt squared is just the variance of Yt squared because this process here has mean zero, right, the constant is zero.
So in this case, the expectation of Yt squared is equal to sigma Y squared, which is, according to what we know, alpha squared times sigma squared Y, because when we take this expectations operator here and put it in the parentheses,
then we have the expected value of Yt minus one squared standing here.
And since we know that the process is stationary, the expectation of Yt minus one squared must also be sigma squared Y.
Plus then two times alpha times the expectation of the product of Yt minus one ut.
Well, the expectation of the product Yt minus one ut is obviously equal to zero, because ut is white noise, so it is not correlated with anything.
In particular, it cannot be correlated with Yt minus one because Yt minus one is just based on information which was available until period t minus one, but ut is a new shock.
So, Yt and Yt minus one and ut are certainly uncorrelated.
So the expectation of this product here must be equal to zero.
And therefore, we're just left with sigma squared u, because the expectation of ut squared is sigma squared u,
which then resides in the equation one minus alpha squared sigma squared Y is equal to sigma squared u.
Now, if alpha is in fact smaller than one absolute value, then we get that sigma squared Y is sigma squared u divided by one minus alpha squared, which is greater than zero, of course.
And this is exactly the same result which we had in equation 17. Just divide this equation here by one minus alpha squared, which is number which is different from one.
And different from zero, so we can divide through by one minus alpha squared, and then we get the familiar result that sigma squared Y is equal to sigma squared u divided by one minus beta squared.
However, if alpha happens to be exactly equal to one absolute value, then we would get that sigma squared u is equal to zero.
Because alpha equal to one absolute value would imply that the term in parentheses here is zero. So zero times something is zero.
So this would mean that sigma squared u is zero. So the variance of the white noise process is equal to zero.
But that cannot be the case because the white noise process was defined to have a positive variance. Otherwise, if there's no variance, this would just be constant of zero.
And that's not a white noise process.
So, the fact that sigma squared u is equal to zero would contradict the assumption that ut is a pure random process.
And therefore, the variance of Yt cannot be constant in this case where alpha is an absolute value equal to one.
And we will come to this later and show that this is indeed the case that the variance of such a process where alpha bar is equal to one is greater than,
sorry, that this variance must be non-constant in the case that alpha bar, that alpha an absolute value is equal to one. So we will show this later.
Similarly, if alpha an absolute value would even be greater than one, then we would have that sigma squared y is equal to sigma squared u divided by one minus alpha squared.
But one minus alpha squared in this case would be negative. So the whole variance of Yt would be negative, which is impossible.
So that's also a contradiction. And therefore, we know from this equation here that in this case where alpha in absolute value is greater than one, the variance of Yt sigma squared y cannot be constant.
And this concludes the proof, of course. So the AR1 process is not stationary if alpha exceeds one in absolute value or is equal to one in absolute value.
So the AR1 cannot be stationary in this case. And that was the proposition that AR1 is not stationary if alpha is equal to or exceeds one in absolute value.
Here's the graphical illustration of the same result. What I have done here is pretty much the same thing that I did already with my series X1 and X5 previously.
I used the same white noise errors to generate two AR1 processes. One AR1 process with alpha equal to 0.9, this is the blue line, and one AR1 process with alpha equal to one.
So one may say the difference is not so big between 0.9 and alpha equal to one. In particular, if you know that the white noise errors are the same for both processes.
But look at what happens to the mean. If you just look at the blue line here, then you might well say it's plausible to think of a constant mean over time.
Essentially, it seems to be that 0, for instance, is quite a good estimate for the mean of this time series process here, because the series very clearly fluctuates around 0 or some value close to 0.
So this series can plausibly be argued as a constant mean over time.
But now look at the Brown series, which is the case alpha equal to one, and see how this develops. It goes up much further sometimes, but then it has a long, long period here where it is basically just dropping down.
Now for this series, it is much harder to say what the mean of the series is.
The series was almost constantly beyond 0, so it was almost constantly positive for the first half of the sample, with some exceptions here, but mostly it was positive so that you would have expected the mean is positive.
And then the series declined and declined and declined and declined without any evidence that the series would recover, which is in fact a property of such periods with alpha, of such series with alpha equal to one, that they drift over time and have no tendency to return to the previous means or previous sample values.
So that, I hope, illustrates somewhat that a process with alpha equal to one does not have a time constant mean and actually doesn't have a time constant variance.
Any questions up to here?
Raise your hand if there are.
We have completed the case of the AR1 process and come now to the case of an AR2 process, which is of course a little more complicated because we have now two lagged y terms in the definition of yt.
So the AR2 is yt is equal to a constant nu plus alpha1 yt minus one plus alpha2 yt minus two plus ut, which again shall be white noise.
So let me write this immediately in a leg operator notation as an equation 22, one minus alpha1 l minus alpha2 l squared times yt is equal to nu plus ut.
So I have simply moved the alpha1 yt minus one and the plus alpha2 yt minus two over to the other side.
And then I write the yt minus one is l times yt and I write the yt minus two is l squared times yt so that I can have this leg polynomial here.
This is, as you see, a polynomial in the leg operator in l.
So I write it as alpha of l, alpha of l is a shorthand notation for this leg polynomial, which is one minus alpha1 l plus alpha2 l squared.
Now you know from basic algebra that polynomials can be factored. So they can be written as a product.
For instance, we know that one minus alpha1 l minus alpha2 l squared can be written as one minus lambda1 l times one minus lambda2 l.
So that would be a product definition of this leg polynomial and this here is a linear definition of this leg polynomial linear in the coefficients here.
Now, in order to find lambda1 and lambda2, I haven't pointed this out yet, but I should perhaps do this, lambda1 and lambda2 are so far unknown numbers, real numbers or imaginary numbers.
This depends. But they are not time series, they are just numbers. And we don't know what their size is, but we know they are related to alpha1 and alpha2.
To find the lambda1 and lambda2 values here, note that the leg polynomial one minus alpha1 l plus alpha2 l squared can also be written as one minus lambda1 plus lambda2 l plus the product of the two lambdas, lambda1 lambda2 times l squared.
So we know that alpha1 is equal to lambda1 plus lambda2 and negative alpha2 is equal to lambda1 times lambda2,
which actually has a relationship to matrix algebra where you know that the eigenvalues of a matrix of say a two by two matrix, call them lambda1 and lambda2 have the property that
Yeah, one, one root of the characteristic equation is equal to the sum of the eigenvalues, namely lambda1 lambda2 so that's some of the eigenvalues and the other one is the product of the eigenvalues.
So that's basically then related to the determinant and to the trace of a matrix.
But it's just if you know these relationships are not necessary to understand them here. What is necessary is that you see that alpha1 and alpha2 are related to lambda1 and lambda2.
Okay.
Substituting this in the first, sorry, substituting the first in the second equation, we then get a quadratic equation in lambda1.
So we try to solve for the lambdas now in order to write the lambdas as functions of the known coefficients alpha1 and alpha2.
So if I substitute this equation here into this equation, then I get a quadratic equation in lambda1, and that would be lambda1 squared minus alpha1 lambda1 minus alpha2 is equal to zero.
And all we need to do now is what you have learned in high school ninth grade or 10th grade or something like that.
So the quadratic equation, and then we get the solution for lambda1 and lambda2.
That's easily done. You can verify this. This is a small exercise here, but that's really a trivial exercise. Lambda1 or lambda2 are given by alpha1 over 2 plus or minus.
So plus would be lambda1 minus would be lambda2. The square root of alpha1 over 2 squared plus alpha2.
Whatever the result here is, and this would be some number, pair of numbers in the space of complex numbers.
They may be real, but they may also be complex depending on whether the term under the root here, whether it's discriminant, is positive or negative.
Now we won't really bother much about the question of whether these are complex roots or not. What is more interesting to us is whether they are in absolute numbers smaller than one.
So assume that this is the case that lambda1 and lambda2 are both smaller than one absolute value.
In this case, we can multiply equation 22, which was this equation here, where the Lag polynomial can actually be written as a product as 1 minus lambda1L times 1 minus lambda2L.
So think about this here in place of this parentheses.
We can multiply this equation here by the inverses of these two Lag polynomials, because as you have learned, for 1 minus lambda1L, there exists an inverse.
And for 1 minus lambda2L, there also exists an inverse provided, as I have already assumed, that lambda1 and lambda2 are both smaller than one absolute value.
The inverse is, of course, 1 over 1 minus lambda1L and 1 over 1 minus lambda2L.
In this case, therefore, multiplying equation 22 by the inverse Lag polynomials, we get yt is 1 over 1 minus lambda1L times 1 over 1 minus lambda2L times nu plus ut.
And this we can reformulate a little bit, because what we have to do is essentially that we get rid of these Lag operators here.
And you know how to do it because the Lag operators here can be developed into an infinite sum, just in the way in which we have done this for the AR1 case.
Only now we have two such Lag operators, which have to be applied successively. So we get not just one infinite sum, but we get two infinite sums, which in this case are running from i is equal to 0 to infinity and j is equal to 0 to infinity times lambda1 to the i and lambda2 to the i times ut minus i minus j.
So that would be the rewriting of the first line by which we eliminate the two Lag operators. Here, in place of the Lag operators, we now have delayed values of the u's here, delayed by i periods and by j periods corresponding to the indices of the two summations in here.
Well, this double sum here is still not in the form in which I would like to have it. So what I do is that I rewrite one of those sums as a finite sum, and I do this in two steps.
First step involves that I combine the i and the j here to a single index k. So I define that k is equal to i plus j.
As you can see here, here we have t and then we subtract i plus j in parentheses from t. So we can also write ut minus k if we define k to be the sum of i plus j.
Question is, how does the sum change when we now introduce the k, the new index, in place of either the i or the j? Let's suppose that you substitute out the j. So j by virtue of this equation here, j is equal to k minus i, obviously, right? j is equal to k minus i.
So we have here ut minus k. Now here we have lambda 2 to the power of j. So for j, now we have to write k minus i, right? It gives me lambda 2 to the power of k minus i here.
The exponent of the lambda 1 is unchanged. Lambda 1 to the power of i, as it is here, we don't substitute out anything. We just substitute out the j. We do not substitute out the i, right?
For the j, we get then k minus i. Now here is another j. So this has j is equal to, well, running from 0 to infinity. So we have to do the same thing here again. For j, we can write k minus i. So k minus i should run from 0 to infinity.
Well, this, of course, implies that k runs from i to infinity. Just if you think of the j being replaced by k minus i, then bring over the i to the other side. So you would have k is equal to i going all the way to infinity, which I have written here.
Okay? So here we are. And now I claim that the sum which has as the first sum an i index going from 0 to infinity and as the second k index going from i to infinity can also be written as a sum where the outer sum is with a k index.
k was the inner index here. Here's the outer index. k now going from 0 to infinity and i going from 0 to k. So the inner sum here is now a finite sum, whereas here it was an infinite sum.
And just for one slide, please believe me that this is correct what I write here to see the other changes I've made in the second equation. And then I will explain to you why this double sum here can also be written as this double sum here.
So what are the other changes I've made? Well, I have replaced 1 minus lambda 1, sorry, 1 minus lambda 1 times 1 minus lambda 2 by 1 minus alpha 1 minus alpha 2.
Which is just what we know from the definition of lambda 1 and lambda 2 that this multiplicative form here is the same as this linearly additive form in here and that then the product of the lambdas is equal to alpha 2 and the sum of the lambdas is equal to alpha 1.
Sorry, the product of the lambdas is equal to negative alpha 2, set to alpha 2, but it's negative alpha 2. Okay, so that's trivial and follows from what we had on the previous page. And then what I would like to draw your attention to is that I can take this finite sum here and say, well, this is a psi k.
Because this finite sum, first of all, this sum is finite because it's just going from 0 to k, so it has k plus 1 components, the sum, and this sum just depends on k. Since i is just an index of summation, right, which we get rid of by summing, so we can say that this inner sum here is sum psi k.
And the question now remains just how do I move from these two sums here, both of them infinite, to these two sums here, where just the outer sum is infinite but as the running index of the inner sum in the previous line and the inner sum is finite.
Well, that's actually not as difficult as it may seem, and I think the easiest way to understand what's happening is if you order elements which are indexed both by i and k,
let's say with these stars here, so think of each star as some element which depends on i and on k, order them such that the i's here are defined the rows and the k's define the columns, okay?
So what happens when we have a sum which is i is going from 0 to infinity and k is going from i to infinity?
What I claim is this double sum here, two infinite sums, just adds the entries here, the start entries row-wise, because this sum I would start with i equal to 0, and for i equal to 0, I would then k, let's go from 0 to infinity, so I would add up this, this, this, and this, and all the way to infinity.
And then I would move to i equal to 1, and for i equal to 1, I would start with k equal to 1, so I would again be in this line here and add up this coefficient as the first term, k equal to 1, and then this, k equal to 2, and so forth, all the way up to infinity.
And here, of course, for i equal to 2, we start at k equal to 2, so again, these coefficients and many more in the future would be added up.
So that's exactly what happens if we enter the entries row-wise.
The second double sum here, which corresponds to my second line on the previous page, adds the entries just column-wise, which is of course the same, right?
So here, I would start at k equal to 0, which is this one here.
If k is equal to 0, i starts at 0, ends at 0, so this would be just i equal to 0, so I would just add this thing here.
And for k equal to 1, I would have that i goes from 0 to k, so k equal to 1 is here, i goes from 0 to k, so I would add these two, and so forth.
For k equal to 2, i would go from 0 to 2, so k equal to 2, I would add these three elements in the column here.
And doing this for our k, let k go from 0 to infinity, I would go through all the columns here, and I would also add up all the elements.
So very clearly, this double sum here is the same as this double sum here, and that's all I had used in equation 25 here, and this corresponds to the first row, and that corresponds to the second row in this representation here.
All right, any questions for the way I dealt with the equation here?
What I would now like to show is that equation 21 is stationary when lambda 1 is smaller than 1 in absolute value, and lambda 2 is also smaller than in absolute value.
So equation 21, 22, was this equation here, or that equation here, doesn't really matter which one I take, but what I want to show is that the AR2 is a stationary process.
If it is true that these lambdas here, which are, as you have seen, related to the alpha 1s and the alpha 2s by virtue of these two equations, or by this equation at 24 here, which gives me how lambda 1 and lambda 2 are defined with reference to alpha 1 and alpha 2.
I would now show that this process 21 is stationary.
And that follows immediately if we can show that 25, the equation we had on the previous page, this representation here, that this is a world decomposition.
So all we need to show is that the psi k coefficients, which I have defined here, that these psi k coefficients are square summable.
That's, as you know, a condition for the world decomposition.
We have an infinite sum here of the psi k coefficients, and we need to show that the squares of the psi k's can be summed up to infinity and yield a finite value.
Unfortunately, there's some math involved here, and I won't go through all the equations in detail. You can do that at home, but there's no other way to show it, but to derive this result mathematically.
So what we have to show is the sum of the psi k's, psi k squares here, the sum of the psi k's squares, infinite sum, as you can see here, is more than infinity.
If the psi k is defined, as I had done it on the previous page, as finite sum over lambda 1 to the power of i lambda 2 to the power of k minus i.
How can I show this? Well, there is a little bit of math, as I say, involved. This comes on this at the previous slide, the subsequent slides.
First, let us just assume that lambda 1 is less than lambda 2 in absolute values, and both of them are smaller than 1.
This assumption here is without loss of generality, because I can call either value lambda 1 or lambda 2, so just take the smaller one and denote it by lambda 1, smaller one in absolute terms.
The larger one in absolute terms is lambda 2, and now I just look at a single psi k and show that psi k squared is less than or equal than this thing here.
So this thing is finite. So first thing I show that psi k squared can have or has an upper bound, which is this bound here.
I just show that this thing is less than or equal than that. You see, I have two less than or equal signs here.
So how to do this? Well, I look at the psi k squared. So this is psi k by definition, and I have to square this.
Squaring a finite sum yields a double sum, which is also finite in both components. So now it is i going from 0 to k and j going from 0 to k.
And of course, lambda 1 i squared is then, sorry, lambda 1 to the power of i is then multiplied by some lambda j to the power of i, which gives lambda 1 to the power of i plus j.
And similarly, lambda 2 to the power of k minus i is multiplied then by some lambda 2 to the power of k minus j. So we get some lambda 2 to the power of 2k minus i minus j.
Well, this is certainly less than or equal than this expression here, where here I have lambda 1 over lambda 2 to the power of i plus j.
Because what I do is just I take this lambda 1 here, and then I take the lambda 2, which has also an i plus j exponent, but with negative sign in it, divide by it.
So get lambda 1 divided by lambda 2 to the power of i plus j, and are left with lambda 2 to the power of 2k, which is this component here.
Okay, now we know that lambda 1 over lambda 2 is smaller than 1 by this assumption here.
So we can, or perhaps, sorry, I should first have mentioned, why do I have a less than or equal sign already here? Because I was introducing the absolute values here.
So this thing is certainly greater than lambda 1 over lambda 2 to the power of i plus j without the absolute signs here.
And then I have another inequality, less than or equal, this double sum here, lambda 2 to the power of 2k, because I know that all these terms here are smaller than 1 or less than or, yeah, actually they are smaller than 1.
So I can actually forget about them and just sum the lambda 2 to the power of 2k's, and then up here.
And here I use some results from analysis that such a double sum is equal to 2k plus 1 squared times lambda 2 to the power of 2k.
And therefore I know that this thing is finite.
Well, how do I know that?
Well, essentially, what I have here is that I have k plus 1 terms in each sum, right? Here are k plus 1 terms, and here are k plus 1 terms.
So I know that this thing is equal to k plus 1 squared times lambda 2 to the power of 2k, essentially using that this thing here is independent of i and of j, so I just have to count terms.
So using this, I can now take the infinite sum over the psi k squared.
So here I let now the sum with index k go from 0 to infinity.
And this thing would be, well, yeah, less than or equal to k plus 1 squared times lambda 2 to the power of 2k by virtue of the inequalities we had on the previous slide.
What I have to do here is that I have to multiply out this expression k plus 1 squared. This gives to me k squared here, 2 times k here, and just a plus 1 here.
Well, and now one can show that this sum here is the same thing as lambda 2 to the power of 4 plus lambda 2 squared divided by 1 minus lambda 2 squared to the power of 3.
And this sum here is, well, sorry, this sum here is equal to 2 times lambda 2 squared divided by 1 minus lambda 2 squared, and the so thing squared.
I show on the next two pages that these relationships here are true, so that this is the result of that sum, and that this is the result of that sum.
What you do know is that this is the geometric sum, where just lambda 2 squared is raised to the power of k, and you see this is an infinite sum here, so this is 1 over 1 minus lambda 2 squared.
These two things here are finite, so the whole thing is finite, and this is what we need to know. So we know that 25 is a world decomposition with psi k being equal to this term here.
And therefore we know that the yt is in fact stationary.
Now, how does the last equality on the previous page follow? Well, I first explained to you why this sum here is equal to that sum here, and you find the proof of that in here.
The way to proceed is rather easy. I have here the sum over i times alpha to the power of i.
Well, what does this mean? This essentially means that this sum here can be split up into infinitely many sums of the type alpha to the power of i, just starting at a later point.
So here the first index is 1, here the first index is 2, here the first index is 3, because if i is equal to 1, then I just have here this sum of terms with 1 times alpha to the power of i.
And this corresponds then to this sum here. But if alpha is equal to 2, then I have 2 times this sum, but starting at i equal to 2, so I have to add another such sum here.
And if i is equal to 3, then I have 3 times the sum of the type alpha to the power of i, so I have to add another term here, just starting at 3.
So this we can rewrite as sums which always start at i equal to 0 here by factoring out an alpha here, an alpha square here, and an alpha 3 here.
So essentially we just have this sum times that sum, and the second one here is the geometric sum, which is just 1 over 1 minus alpha.
And the first sum is just alpha times the geometric sum, so we have alpha times the geometric sum raised to the power of 2, which is then alpha over 1 minus alpha squared.
And this is something, I won't ask you for this in the final exam, but you should keep it in mind because we need this later on.
Along the same lines, we actually have to prove that the sum over i squared times alpha i is equal to this thing here, and I won't go through this in detail.
This you can do at home. I think I have all the steps in here, and it follows precisely the same ideas as this proof here.
When you have verified this down here, you will see that this explains this value as the result of this sum here, so the proof is complete.
Okay, after having done this, we have simple proposition. For an AR2 process, the void decomposition coefficients can be recursively computed from the following recursion.
Psi k is equal to alpha 1 psi k minus 1 plus alpha 2 psi k minus 2.
We will just need two starting values, namely psi naught and psi 1. Psi naught shall be equal to 1 and psi 1 shall be equal to alpha 1.
Okay, so the alpha 1 coefficients are known from our definition of the AR2. For psi naught, I just said that this is equal to 1.
So if I have psi naught and I have psi 1, then from this recursion here, I can compute psi 2. And with psi 2 and psi 1, I can compute psi 3 and so forth.
Why is this true? This is really helpful proposition. Well, psi k is, as we have seen, this finite sum here, which implies already that psi naught is equal to 1, and that psi 1 is equal to lambda 2 plus lambda 1.
Just evaluate this sum here for k equal to 1, and lambda 2 plus lambda 1 is equal to alpha 1. So this is this thing here.
So what do we have to show? We have to show that psi k minus alpha 1 psi k minus 1 minus alpha 2 psi k minus 2 is equal to 0 to prove this recursion here.
Well, the alpha 1s and the alpha 2s are related to the lambdas in the following way. So we can equally well write psi k minus the sum of the lambdas times psi k minus 1 plus the multiplication of the lambdas times psi k minus 2 is equal to 0.
Now we use the definition of psi k and we get again some formula, but easy to verify at home. Psi k minus the sum of the lambda psi k minus 1 plus the product of the lambda psi k minus 2.
Well, it can now be written using the definition of the size in this form here, just shifting the k index up here in the sum accordingly and the exponent here accordingly.
And then you rearrange terms and again I asked you to verify at home that this is all correct and eventually you arrive at that this is lambda 2 to the power of k minus lambda 2 to the power of k, so it's equal to 0 and that completes the proof.
Okay, this was the worst of mathematics for this lecture and perhaps for many lectures to follow. It's a little annoying perhaps and a little cumbersome to work with these types of sums.
It happened so much more in the next lectures. I do stop the lecture here for a five minute break, and then I continue with the insights that we gain from what we have already derived on the AR2 process and we'll also supply with some empirical
examples later on, not artificial data, but then really empirical examples. So see you again in five minutes please.
Good. Let me resume the lecture here. Any questions so far?
Please raise your hand.
No questions? Okay.
Good. So we have proven that if lambda 1 and lambda 2 are both smaller than 1, an absolute value, that then the AR2 process is a stationary process.
This time, this proof, we did not proceed by checking that the mean is constant over time and the variance is constant over time and the coverances do depend only on the time difference but not otherwise on time, but rather we have just made use of the properties
about decomposition and shown that the representation of the process we derived is actually about decomposition, which is, even though it seems rather complicated, easier for such processes and the more complex processes which follow when we have an ARP than using the standard proof.
Okay. Now, the lag polynomial which defines our AR2 process is, as you recall, 1 minus alpha 1L minus alpha 2L squared, and we can factor it in this way as we have said, so as 1 minus lambda 1L times 1 minus lambda 2L, and the lambda are just simple functions of the alphas here.
Now, obviously, this lag polynomial is invertible. If the lambda 1 and the lambda 2 are both smaller than 1, an absolute value, because we know for a lag polynomial of 1 minus lambda 1L, we have an easy way to find the lag polynomial as 1 over 1 minus lambda 1L, which then can be developed into the geometric sum.
And the same thing for this here.
Obviously, this inversion of the lag polynomial is only possible if the lambdas are smaller than 1 and absolute value, because only then do these inverses exist.
Here's the inverse again for some lambda i, either i is equal to 1 or i is equal to 2, and you know the inverse is always this expression here, which is the geometric sum.
Because only if these things are smaller than 1 absolute value does the infinite sum converge.
Now, since we know that the lag polynomial alpha of L can be written as 1 minus lambda 1L times 1 minus lambda 2L, it follows that alpha evaluated at L, the lag operator, being equal to 1 over lambda 1 is equal to 0.
If for L, I substitute in 1 over lambda 1, then I have lambda 1 times 1 over lambda 1, which is equal to 1, so 1 minus 1 is 0.
So apparently, the lag polynomial has a 0 at point 1 over lambda 1. And similarly, it assumes value 0 at 1 over lambda 2.
So the inverses of these lambdas here are what is called the root of a lag polynomial. Recall that the root of a polynomial is always the point at which the polynomial assumes value 0.
So the lambda 1 and the lambda 2 are the coefficients which are derived from alpha 1 and alpha 2, and their inverses are the roots of the lag polynomial alpha of L.
So rather than saying that the series is stationary for lambda 1 being smaller than 1 in absolute value, or lambda n to lambda 2 being smaller than 1 in absolute value, we can also say that a lag polynomial alpha of L is invertible if all its roots are greater than 1 in absolute value.
So the root is always the inverse of the lambda coefficient.
The problem is that in time series literature, sometimes the condition is used that the lambdas are smaller than 1, and sometimes the condition is used that the root of a lag polynomial is greater than 1.
And then what I would like to alert you to is that this is the same thing, right? So the lag polynomial alpha of L is invertible, which implies that the process is stationary if all the roots are greater than 1 in absolute value.
All the roots of the lag polynomial are greater than 1 in absolute value. The root is, as I say, the inverse of the lambdas.
This definition is true for all lag polynomials, not only for lag polynomials of degree 2, because very clearly if you have a general lag polynomial alpha of L with more than two lags here, then we know any polynomial can always be factored.
It has a number of roots, and it is always true that when the roots are greater than 1, then the polynomial can be inverted because then we can invert all these factors, 1 minus lambda 1L, 1 minus lambda 2L, 1 minus lambda 3L, and so forth.
So rather than looking at an AR3 or ARt process, let's go back to the AR1 case, which was particularly easy. In this case, the lag polynomial is 1 minus alpha L, as you recall.
And the root of this polynomial is the solution in L now of this equation here. So the root of a polynomial is the point where the polynomial takes on value 0, so 1 minus alpha L must be solved for L.
Well, that's very easy. L is then 1 over alpha in this case. So the invertibility condition is that 1 over alpha absolute value is greater than 1, or that alpha bar, alpha and absolute value is smaller than 1. So that's an equivalence.
Either the inverse is greater than 1, or the value of alpha itself and absolute terms is smaller than 1.
Now, just as a remark, which you should know from your math lectures, a polynomial of degree P always has P roots in the space of complex numbers.
The P roots are not necessarily distinct roots. There may be roots which have double or even triple or even more multiplicity, but there are always P roots for such a polynomial, some of which may coincide with each other.
Now let's explicitly compute the moments of the AR2 process. We had equation 25, which I will show you again briefly here.
This was equation 25. After some messy rearrangements, we came to the conclusion that yt is equal to nu divided by 1 minus alpha 1 minus alpha 2 plus double sum here over the ut minus k's, where actually this double sum is not a double sum anymore if we collect this inner sum here and say that this is just psi k.
So then we just have an infinite sum of psi k times ut minus k.
From this representation of the AR2 process, yt is equal to this, we immediately see that the mean of the AR2 process is nu divided by 1 minus alpha 1 minus alpha 2, because the expectation of this term here is always 0, since the expectation of u is equal to 0.
So very easy.
Where are we? Here. The expectation of yt is nu divided by 1 minus alpha minus alpha 2, and I define this to be equal to mu. So mu again gives me the expectation of yt.
And for the variance, then we will have to look at the sum, the infinite sum of the psi k squares.
Well, this is sigma squared u times the sum over the psi k squares, which is then set to be or defined to be sigma squared y.
And the covariance is, of course, just double sum then over psi k's and psi j's and correspondingly ut minus k's and ut minus tau minus j's multiplied with each other.
And most of these products here have an expectation of 0, namely any time when t minus k is different from t minus tau minus j, and the expectation of this product is 0 because u is right noise.
But in the specific case in which k is equal to tau plus j, we would have the expectation of ut minus k squared.
And this expectation is of course sigma squared u, so that we arrive at sigma squared u times an infinite sum of k going from tau to infinity over psi k minus tau.
And observe now, this expression here does not depend on t anymore, it just depends on tau. So it just depends on the time difference between two points in time, but not on the exact timing on when t and when s is.
Rather, it depends just on the time difference. So this is the gamma of tau, the autocovariance at lag tau. And it is independent of time, just depends on the time difference.
So again, of course, we see that the process is stationary.
From these fairly easy calculations, we can only derive a proposition which I'm going to prove a little well in the two lines below.
The proposition says for tau greater or equal to 2, the covariance follows a simple autoregression. So the covariance of an AR2 process follows a simple autoregression.
Namely, the covariance at lag tau is equal to alpha 1 times the covariance at lag tau minus 1 plus alpha 2 times the covariance at lag tau minus 2.
So this proposition corresponds exactly to the same proposition we've had in the AR1 case, where we saw that in the AR1 case, the covariance of the AR1 is a simple autoregression of first order.
And here we see that the covariance is a simple homogenous autoregression of second order. And that is actually true for any autoregressive process, that its autocovariance function
also has the autoregressive structure which inherits from the representation of the data in the time domain.
So we see that we have the same type of autoregression in the covariances as we have it in the observations.
The observations had the property that yt was equal to alpha 1 yt minus 1 plus alpha 2 yt minus 2 plus constant plus noise, which we don't have in here.
That implies that here we have a homogenous difference equation, whereas in the observations it was not homogenous.
But otherwise, it's exactly the same type of autoregression which we have in the observations as we have it here in the covariances, which is a very convenient property.
Now I proved this just for the case of mu equal to 0, which is just for simplicity, it's very easy to write the proof with mu different from 0, but it's more notation.
So what do we have to do? We just multiply our initial representation of the AR2 process, so the equation yt is equal to alpha 1 yt minus 2, yt minus 1 plus alpha 2 yt minus 2 plus constant plus noise, but constant is not here anymore.
This we multiply by yt minus 2 on both sides of the equation. So we get yt times yt minus tau is equal to alpha 1 yt minus 1 times yt minus tau plus alpha 2 yt minus 2 times yt minus tau.
So both sides I have just multiplied by yt minus tau plus ut times yt minus tau.
Now, if I just take expectations of this equation here 28, then I already have an equation 27. Why is this? Well, obviously the expectation of yt times yt minus tau is equal to the covariance if the mean is 0, right?
Here the same thing, the expectation of yt minus 1 times yt minus tau is the auto covariance for leg tau minus 1. On this side it was the covariance for leg tau.
Here it's the covariance for leg tau minus 1. And here, same thing, t minus 2 and t minus tau is the covariance at leg tau minus 2.
So we can get here the expectation of this product to be the auto covariance at leg tau minus 2, and this is being multiplied by alpha 2.
Why is this thing here? Why does this have an expectation of 0? Well, because ut is white noise.
And white noise is uncorrelated with everything which happened previously, yt minus tau, for tau being greater than or equal to 0, realized itself before the ut was realized, so ut is independent of yt minus tau.
Therefore, the expectation of this thing here is 0 and drops out, and then we have proved equation 27 here.
As an exercise, you can show that 27 also holds in the case of mu different from 0.
When this holds for the auto-covariance, obviously it also holds for the auto-correlations. So we have the simple corollary that for tau greater than or equal to 2, we have the simple auto-regression of the auto-covariance,
rho tau is equal to alpha 1 rho tau minus 1 plus alpha 2 rho 2 minus 2.
And the proof is obvious, we just have to divide equation 27 by the variance gamma of 0.
So we know now that the auto-correlations follow the same auto-regressive law as the time series itself, which I have reproduced here again for convenience equation 21, which we know already.
21 has an inhomogenous part, which is nu plus ut, so this nu here plus ut.
This inhomogenous part we have gotten rid of when we just look at the auto-correlations. So we have actually not lost much information, but we just look at the auto-correlation structure rather than at the data structure.
You may recall from your math lectures that the solution to a second order difference equation can display oscillations.
So it is interesting for any kind of cyclical analysis when you want to analyze business cycles, for instance, you always need at least a difference equation of order 2 in order to get non-trivial oscillations in your data.
A difference equation of leg 1 of order 1 just gets trivial monotone converges to equilibrium value. There would be no oscillations in there, but with a difference equation of order 2, we may get
oscillations and the same holds true for difference equations of higher order. So linear, stochastic, second order difference equations are actually interesting for any kind of cyclical analysis in economics.
Now, what we have learned now is that we can see such cycles not only in the observations and the actual data that we observe, but we may also see them in the auto-correlations.
We should see them just the same way actually in the auto-correlations because the auto-correlations follow the same type of autoregressive law, which generated the data.
Yeah, so that's the point I made here. The solution to the homogenous part of 21, so just this alpha 1 yt minus 1 and alpha 2 yt minus 2, that's the homogenous part here, displays damped oscillations if the roots of the lambda 1 and lambda
2 of this characteristic equations are a pair of complex conjugates. So if they are complex numbers, if they are real numbers, then we have to do a monotone convergence. But if they are complex, the roots of the characteristic equation, then we can get damped oscillations,
damped meaning that actually the oscillations have a decreasing amplitude over time and converge to a fixed equilibrium value, but with a stochastic shock like the ut, it can be, or it is actually the case, that the shock moves the system away from the equilibrium ever once again, so that the oscillations never
completely converge, but rather that convergence means that at the steady state, we always have oscillations of the same amplitude.
So yeah, complex roots means of course that the discriminant here, the term under the square root is negative, right? So we have oscillations with alpha 1 over 2.
Square is less than negative alpha 2.
As I have already said, for an AR1 model, we would not get interesting oscillations. We can at most get primitive Bing-Bang behavior if alpha is negative, because for AR1, we would have yt is equal to alpha yt minus 1 plus noise.
So if alpha is negative, we would find that alpha 1 is positive when alpha 2 is negative, sorry, alpha t is positive whenever t minus 1 is negative, but the positive alpha t implies again a negative alpha t plus 1 in the next period, because the alpha always changes the sign.
That's not really a very interesting cycle. This is something which is rather trivial. If alpha is positive, then we actually have more convergence. In any case, typically we would say that one process is not a good process to depict something like business cycle or other types of cyclical phenomena in economics.
Now, our corollary on this page here shows us that we may have systematic oscillations in the time series if and only if we also have the same systematic damped oscillations in its autocorrelation function.
So rather than looking at the data, we may actually just look at the autocorrelation function. And what is the advantage of this? Well, look at this time series here, which I have taken from the previous works of time series US data quarterly,
which I have uploaded to Steene, and this is the growth rate of quarterly GDP starting in 1948 and going because this was part of a publication of mine until 2007. So what we see in this graph here are oscillations in the growth rate of
real per capita GDP in the United States of a rather long horizon of more than 50 years, almost 60 years.
Okay, if you look at these data, by the way, in the work they are called d GDP because I couldn't use a delta as a symbol. So the d is the same thing as this delta here.
And if you look at these oscillations here, it is rather difficult to discern some type of cyclical behavior, some type of cyclical phenomenon in the data.
But now look at the corbellogram of the same series. So this is the corbellogram of the d GDP.
I did this in a different program, not in Eevee, but in Gauss. So the corbellogram is not horizontally rather than vertically organized, but otherwise it's the same type of representation that I have already explained to you in Gauss.
The dotted lines here are two standard errors plus minus two standard errors. Here's the zero line, and here you see the size of the first order autocorrelation, estimated first order autocorrelation, estimated second order autocorrelation, estimated third order autocorrelation, and then it's going here in the negative domain.
So very clearly, and you see first, positive autocorrelations, then negative autocorrelations, some positive autocorrelations, again negative autocorrelations, again positive autocorrelations, again negative autocorrelations, again positive autocorrelations, negative autocorrelations, positive autocorrelations.
So I was evaluating here 36 autocorrelations, which is quite long, right? And very clearly we see oscillations in the autocorrelation function here. We see oscillations in the corbellogram, which are not that clear in the time domain representation of the data.
Going to see a cycle in here is difficult, right? But to see a cycle in this graph, which uses exactly the same data, is much easier.
Of course, one may now again ask the question, well, are these things here significant? Clearly, many are not significant, and some are just barely significant, like this one here, or this one is significant.
Again, so not surprising that there is not so much significance because, as you see, the time series itself does not represent a pronounced cycle. It's not very easy to see it.
But there is something in it which resembles cyclical movement if you just take the point estimates here.
And that is already helpful for the work, for analyzing this time series in perhaps other ways and with instruments which we haven't developed yet in order to find out what the cycle is like.
It isn't truly a cycle or is perhaps something spurious, but it seems to be rather regular here.
Further analysis would actually show that the AR2 specification for this particular series, DGDP48, right, may indeed be a reasonable specification.
I won't go into the details here, just leave it at showing you the cochlear lobe of this series, but if you continue working with that, a user AR2 specification, then that's not, well, that's defensible.
Okay, you can do that.
Good. Here I show you a different time series from the same data set, which is data SPC48, so that's the stock price index in the United States, namely it's the Standard and Poor's 500 stock price index, also taking over a span of almost 60 years,
also in quarterly data, so I think I have used always end of quarter value of the Standard and Poor's stock index, and I have converted this into consumption units to have some type of real stock price index because obviously stock prices are recorded
in a nominal way and there is no deflator available in the system of national accounts for stock prices, where you have to do the conversion into real value in some mode which is economically convincing.
So the technique I used here was to convert it into consumer units, basically measuring how many representative consumer baskets can a consumer buy for the value of the Standard and Poor's index in a particular quarter.
Again, the time series is in the data set denoted with a D rather than with a delta, I've noted this down here, so DSPC48 is the series which denotes the difference, the first difference of the stock price index starting in 1948.
And again, looking at the mere time series here makes it difficult to discern a sector.
So let's look at the covalogram, and if we look at the covalogram, then we see again some evidence of oscillations in the covalogram.
We start again with positive autocorrelation for the first legs, that's going down to negative autocorrelations. Well, something positive, so clear what happens here, positive again, negative, positive, negative, little bit of positive, little bit of negative, positive again.
So it's less regular than for the GDP, but you may still think, seems to be that there is something cyclical even in the growth rate of the stock price index, which is quite interesting and perhaps even surprising because typically
people from finance would say that the growth rate of the stock market index is something which is not predictable, where there is no information in it, that stock prices follow a random walk and changes in the stock price index are not predictable.
This data here may actually suggest that this is not true, that even stock price growth rates, there is some predictability.
Unfortunately, when you do further analysis on this series here you would find that the sort of dynamics in the covalogram is not well captured by an AR2 process or even higher processes.
So it's much harder to come up with a good model for the stock price index or the change in the stock price index, then it is to come up for a good model of this type of covalogram, which we have found here.
Anyway, we're not doing business cycle analysis here but just going into time series methods and one method is to look at the covalogram at the structure of auto correlations rather than looking at the observed data.
Any questions for the AR2 case?
I don't see any, so let me now cover the general case, which is ARP, which basically follows the same lines as the analysis of the AR2.
So an ARP process would have p lagged values of the endogenous variable. So only change is that now we have alpha1 yt minus 1, alpha2 yt minus 2, plus plus plus, alpha p yt minus p.
Or equivalently, you may just say that the lag polynomial is of a higher order. So we would have alpha of l times yt is equal to nu plus ut as before, so you can always write the AR process and lag operator in such a way that you put the lag operator times yt on the left hand side of the equation
and on the right hand side comes whatever is exogenous or unexplained, maybe a constant plus the ut.
And the difference now is that the alpha of l is not a quadratic polynomial anymore, not of order 2, but it is now a polynomial of order p.
So that's not much of a change. p was 2 before, now p is something other or is more general than that, and we can just look how things change when we have a more general p.
Well, first definition, an ARP process like the one specified in equation 31 is called stable if all the roots of the equation alpha of l equal to 0 are greater in unity and absolute value. So if it's like polynomial is invertible.
Right? So if this alpha of l can be inverted, then the series is stationary because what happens is we have to divide this thing here just by 1 minus alpha of l or no, actually by
alpha of l inversely, we have to multiply by alpha of l inversely and the alpha of l inversely, we can write as that was where I was aiming to 1 over 1 minus lambda 1l
times 1 minus lambda 2l times 1 minus and so forth lambda pl, then we can develop all these inverses of 1 minus lambda il into geometric sums and then we get huge expressions and they more or less explode in terms of notation
or don't become infinite. All of that can be solved, but I won't do it with you because I think you have understood the principle already with an ARP 2 process.
Now, a polynomial of order t, of course, has always p roots, rather than the two roots we've covered before.
Those roots may not be distinct. So, for instance, if you look at an ARP 2 polynomial again, let's say at the polynomial 1 minus 2 alpha l plus alpha squared l squared, then this would be factored as 1 minus alpha l times 1 minus alpha l.
And obviously there are two identical roots now, namely 1 over alpha is the root both for this part of the polynomial and for this part of the polynomial regardless of whether you replace the l here by 1 over alpha
or whether you replace the l here by 1 over alpha. In both cases, you would find that the leg polynomial suits the value 0. So, the root 1 over alpha has multiplicity 2 and that can easily happen in a polynomial of order p
that the roots are not distinct, not all of them distinct, but some of them actually pop up many times or more than one time. So, there are multiple occurrences of the same root, which is then that we say these are actually different roots, but with the same multiplicity.
They are not different in the sense that they assume different values, but quite to the contrary, they assume the same value, but they can be introduced at different positions in the leg polynomial, and at each position, they would make the leg polynomial equal to 0. This is what I meant by different.
Also, some of those roots may be complex valued.
Since the polynomial itself has only real coefficients, the complex roots can only occur as pairs of complex conjugates. So, it can only be
a complex number if there is the complex conjugate also as a root in this polynomial, such that the two of them, when we multiplied,
cancel as complex numbers and become in their result a real number again. So, for instance, if we have a leg polynomial of the form 1 plus alpha squared L, then you can factor this as 1 minus alpha iL times 1 plus alpha iL.
This is again a leg polynomial with the roots i over alpha for L1 and negative i over alpha for L2. So, this is a pair of complex conjugates which we have here because the imaginary part i over alpha is just the negative of the imaginary part here, negative i over alpha.
Well, a stable ARP process is stationary. So, if you can invert the leg polynomial, then the ARP process is stationary.
The converse is unfortunately not true. The stationary process may be unstable.
Okay, right. Look at the proof of this proposition here. So, suppose that we have lambda i as the inverses of the roots of an invertible leg polynomial. So, all the lambda i's are smaller than one in absolute value and suppose initially that all the roots are real.
So, in this case, we can invert the leg polynomial factor-wise. Yt would be alpha of L inverse times nu plus ut.
So, that would be the same thing as nu divided by alpha of 1 because the nu is never shifted in time. It's constant in time, so you can just replace the L by 1.
And here we would have ut divided by the product of 1 minus lambda iL, right, because the product of this thing here is just the leg polynomial of L.
So, we have nu over alpha of 1 divided by plus 1 over this product here, now starting at i equal to 2, right. So, the first of these terms here, I have now separated from the other terms and I have developed this into this sum here using the properties of the geometric series.
So, this is why the ut here becomes an infinite sum of the ut's.
And I can do this many times in a row so that eventually I arrive at nu over alpha of 1 plus the sum of some psi j's times ut minus j.
All we have to show is now that the sum of the squared psi j's coefficients, the infinite sum of these squared coefficients, is smaller than infinity so that this sum converges because then 32 is a volt representation with a constant deterministic term, therefore Yt is stationary.
This claim we have already shown in the AR2 case in the beginning of this lecture. We can now approve the general case just by induction, we don't need to go through as many formulas as before.
So, for simplicity, assume that the nu is equal to 0, and then we have 1 minus lambda L times Yt is equal to sum phi j's, ut minus j, all of them sum from today up to the most distant future, so up to infinity, with lambda in absolute value being smaller than 1.
Well, if this is the case, then I can invert this leg polynomial, 1 minus lambda n, and we'll write this in the following form, 1 minus lambda n inverse times the sum here.
Well, and now if I take the 1 minus lambda L inverse C and develop this into the geometric sum again, then of course I will get some psi j ut minus j summed over all periods from j equal to 0 up to infinity.
What we have to show is that if the phi j squares are absolutely summable, so if their sum is smaller than infinity, then it follows that the psi j squares are also absolutely summable, so the sum of their squares is also equal to infinity, because here we move from the phi j's to the psi j's.
So if this is true for any lambda L, so 1 minus lambda L polynomial, which we have now in inverted, then it holds for all the 1 minus lambda L polynomials.
Well, and here comes the derivation of the psi j's, which looks perhaps messy, but actually it is not so very difficult. What we have to do is that we develop this thing into an infinite sum.
So the infinite sum is here with a second sum not having changed from the first line. So essentially we have a double sum here and the lambda i Li has been moved in here
to make, for instance, from ut minus k, as in the previous equation, you make ut minus i minus k, because there is a lag operator shifting the ut minus k by i periods, so this gives ut minus i minus k.
And then the lambda i and the phi k's, you find them just here. And then again we substitute, for instance, we substitute in a j, the j is equal to i plus k, but you have minus i plus k, so we can make this ut minus j, and the k here is then of course also just j minus i.
So we still have a double sum here, but with this sum we can say that we just look at i is equal from zero to j for lambda i phi j minus i
so that this can be used again as psi j or defined to be psi j. So what I do here is again see I move the inner sum to become the outer sum and then I change the order of
summation from row wise to column row wise, as I have explained this before, so I come to this expression here and I have just psi j times ut minus j sum of all j's from zero to infinity.
Yeah, and well now we have to show that these psi j's are equal, and I won't go through the math here in detail, you can check that at home and that this is true that I can show that the psi j squared less than or equal than this thing here.
There's an equality fine side missing here as I just see, so I will change that and then show that the sum over all those psi j squares is less than
this expression here, and therefore less than infinity. And by having done this we can complete the proof, because if some of those roots are now complex valued the inversion of a single factor will always lead to complex valued psi j's, or will in general lead to complex valued psi j's.
So essentially what we have to do is that the same equations or what we have to recognize is that the same equations and equations also hold if the psi j is replaced by some well absolute value of the psi j's to make a complex number something real, right?
So when subsequently the complex conjugate factor is inverted all the psi j's will again be real and therefore again the square of these things here implies that the coefficients are square summable so that the value of the sum is smaller than infinity and therefore any stable ARP process is stationary.
Good. And here I stop for today's lecture. Please make sure you go through the math at least once to understand that the proofs are correct.
And more important perhaps than understanding the exact math is that you understand the concepts behind and the idea behind that the stationarity properties of ARP processes
always depend on the roots of the leg polynomial and that whenever all the roots are greater than one in absolute value, then we are safe to conclude that the ARP process is indeed a stationary process.
Therefore, currencies and means will be time independent and that we can then verify in the next lecture.
Any questions so far?
I don't see any so then thank you very much for your attention and see you again next week.