Time Series by James D. Hamilton
This is the introduction for Time Series. I will finish this as I work through the book. So far I’m on page 2 so it’s going to take some time to write this page. Be forewarned, this is going to take some time. Also this will be written in conjunction with python as the programming language to express the intuition behind the math explanations.
… to be continued
Chapter One: Difference Equations
I’m not really sure where other books start but a brief review seems to show it’s all over the place. I think Hamilton’s work suggests that it jumps right in to the heart of the matter on a fundamental basis.
To stare we look at difference equations which are at the heart of differential equations and calculus. Basically we can look at difference equations as a shift forward in time or backwards. We can also look at the average of the two. Hamilton seems to move between the two.
The first example starts with a first-order difference equation with a backward view:
$$ y_t = \phi y_{t-1} + \omega_t $$
In this equation $y_t$ is related to its value one perior prior ($y_{t-1}$) based on the standard linear equation of $y = mx + b$. We can move forward in time and represent the value of $y$ at any time in the future time $t$.
$y_0 = \phi y_{-1} + \omega_0$
$y_1 = \phi y_{0} + \omega_1$
$y_2 = \phi y_{1} + \omega_2$
…
$y_t = \phi y_{t-1} + \omega_t$
Now, if we know the starting value of $y$ on date $t = -1$ and the value for $w =$ for dates $t = 0, 1, 2, …$ then we can calculate any future period. This is done through the process called recursive substitution.
$$y_1 = \phi y_{0} + \omega_1 = \phi (\phi y_{-1} + \omega_0) + \omega_1$$ $$y_1 = \phi^2y_{-1} + \phi \omega_0 + \omega_1$$
This continues for $y_2, y_3 … y_t$ with the generalized formula written as:
$$y_t = \phi^{t+1}y_{-1} + \phi^t\omega_0 + \phi^{t-1}\omega_1 + \phi^{t-2}\omega_2 + … + \phi\omega_{t-1} + \omega_t$$
Note that my initial confusion with this formula is that $\phi$ itself remains a constant. Rather the power to which it is raised depending on the time periods changes. So $\phi$ one period from is the same as $\phi$ today.
Example:
There are 1,000 fish stocked in a pond every year and there is a 25% chance of the fish surviving another year. How many fish will be in the bond each year and what will the population of fish be many years from now?
$$y_{n+1} = 1000 + 0.25y_n$$
$y_0 = 1000$
$y_1 = 1000 + .25(1000)$
$y_2 = 1000 + .25(1000 + .25(1000))$
$y_3 = 1000 + .25(1000 + .25(1000 + .25(1000)))$
$y_4 = 1000 + .25(1000 + .25(1000 + .25(1000 + .25(1000))))$
which equals
$y_0 = 1000$
$y_1 = 1000 + 250$
$y_2 = 1000 + 312.5$
$y_3 = 1000 + 328.13$
$y_4 = 1000 + 332.03$
$y_5 = 1000 + 333.01$
$y_6 = 1000 + 333.25$
this is a geometric sequence so can be written as
$$ y_n = 1000 \frac{(1 - .25^n)} {(1 -.25)}$$
so say in 1,000 years there will be 1,333.33 fish in the pond.
$$ y_n = b \frac {(1-r^n)} {(1-r)} + r^n y_0$$
In Hamilton’s book he uses the example of Goldfeld’s estimated money demand function for the U.S. where
$$m_t = 0.27 + 0.72m_{t-1} + 0.19I_t - 0.045r_{bt} - 0.019r_{ct}$$
We can rearrange the equation so that $\omega_t = 0.27 + 0.19I_t - 0.045r_{bt} - 0.019r_{ct}$ and simplifies to
$$ y_t = .72y_{t-1} + \omega_t$$
where $w_t$ is considered a scalar and $\phi^t = 0.72$.
Dynamic Multipliers
So the first question asked by Hamilton is what happens if we get a one-time bump in the $w_0$ amount?
Let’s look at the fish question. So instead of $1000$ fish being added this year there were $1010$ fish added. Oops by the fish and game. The question is what happens to the population of fish? (Note in Hamilton’s example using Goldfeld’s model it’s a one-time bump in one of the variables that made up $\omega_t$.)
First we assume that $\phi^{t-1}\omega_1 + \phi^{t-2}\omega_2 + … + \phi\omega_{t-1} + \omega_t$ remain constant because they already happened in the past. Rather we are concerned with what occurs now and in the future.
We are assuming we know $y_t$ and $\phi$ and looking for the change of $y$ with respect to the change of $\omega$. How does the $10$ additional fish impact our model.
$\Delta \omega = 10$, $\phi = 0.25$, and $y_{init} = 1000$.
$y_{t+1} = 1000 + .25 \times (1,000) + .25 \times (10) = 1,252.5$
$y_{t+2} = 1000 + .25 \times (1,252.5) = 1,313.125$
313.125 - 312.50 = .625
$= \Delta \omega_0 \times \phi^2$ or $10 \times (0.25)^2 = 0.625$
$y_{t+1} = .25 \times 1,000 = 250$
$y_{t+1} - y_t = 2.50$
$y_{t+2} - y_{t+1} = 1$
$\frac {\delta y_{t+1}} {\delta \omega_0} = \frac {2.5} {10} = .25 = \phi^1$
What confuses me is that we are looking at 3 periods of time. We want to know the future value based on $t$ number of steps forward. We calculate that by finding the change of $y_t$ by incrementing $\omega_{t-1}$ by some amount.
Second when looking at the impact of the “impulse” the graph shows the impact over time. In our case it decays to $0$ over a short period of time.
In the example above $y_{init}$ increases by $\Delta \omega$. So the change in $y$ one period forward is really $\phi \times \Delta \omega$. That’s why $\phi$ is called the dynamic multiplier. In a one step change it’s not really dynamic. However if we consider a 2 period change then it’s not $\phi$ but rather $\phi^2$. Thus the multiplier is dynamic over time.
Basically what we calculated was a impulse-response function because $\omega$ shifted only once.
If you look at this and consider $w_0$ changing then $y_t$ will change based on the ratio given by $\phi$. A value less than 1 but positive will shrink. A positive value above 1 will increase exponentially. A negative sign will alternate and will converge if less than zero but not less than -1. Otherwise it will expand exponentially in an alternating fashion.
In the fish example the population converged to $333.333$ because it was between 0 and 1.
Consider per Hamilton’s book the Goldfeld model where national income increased by 1 “unit”, what will money demand be in 2 periods from now?
$$ \frac {\delta m_{t+2}} {\delta I_t} = \frac {\delta m_{t+2}} {\delta w_t} \frac {\delta w_{t}} {\delta I_t} = \phi^2 \frac {\delta w_{t}} {\delta I_t}$$
Plugging in the numbers for $\phi$ and $I$ we get money demand in two periods will be equal to $(0.72)^2 (0.19)$ or $0.098$.
What is important to note is that the multiplier depends on the number of forward periods, which in the above example was 2 and thus the multiplier was squared. Because the multiplier is between $0$ and $1$ it will converge to $0$.
Another way to think of such a change is a transitory change. In this case, the impact will diminish over time.
Now what if Income changed over more than 1 period or let’s say $j$ periods. So instead of being transitory, it has a long-run effect. The formula for this is
$$\lim_{j\to\infty} \Big[\frac {\delta y_{t+j}} {\delta w_t} + \frac {\delta y_{t+j}} {\delta w_{t+1}} + \frac {\delta y_{t+j}} {\delta w_{t+2}}+ … + \frac {\delta y_{t+j}} {\delta w_{t+j}}\Big] = \frac {1} {(1 - \phi)}$$
In our fish population model instead of 1,010 fish added one year, we increase the new fish each year to 1,010. In this regard the impact is?
$\frac {10} {1 - .25} = 13.3$ This represents an almost $1%$ shift versus a temporary $.2%$.
So a 1 time shift increases by 2.50 and decays but a permanent shift gives you a
So back to the Goldfeld model instead of 1 period shift we have a long-run change of $\frac {0.19} {(1-0.72)} = 0.68$ In otherwords instead of a 1 time shift but a permanent shift the increase in demand will be $0.68%$ versus $.0098%$.
pth-Order Difference Equations
(You can work through this two ways. Hamilton starts with the matrix version)
So far we’ve just talked about a first-order difference equation. Now we are expanding to incorporate more previous periods or lags.
$$ y_t = \phi_1 y_{t-1} + \phi_2 y_{t-2} + \phi_3 y_{t-3} … + \phi_p y_{t-p} + \omega_t $$
Using matrix algebra we can simplify the presentation. First we can group the lagged $y$ variables at:
$\xi_t = \begin{equation*} \begin{bmatrix} y_t\ y_{t-1}\ y_{t-2\ …\ y_{t-p-1} } \end{bmatrix} \end{equation*}$
For $\phi$ we do something a bit interesting:
$F = \begin{equation*} \begin{bmatrix} \phi_1 && \phi_2 && \phi_3 && … && \phi_{p-1} && \phi_p\ 1 && 0 && 0 && 0 && 0 && 0\ 0 && 1 && 0 && 0 && 0 && 0\ 0 && 0 && 1 && 0 && 0 && 0\ … && … && … && … && … && …\ 0 && 0 && 0 && 0 && 1 && 0 \end{bmatrix} \end{equation*}$
$v_t = \begin{equation*} \begin{bmatrix} \omega_t\ 0\ 0\ …\ 0 \end{bmatrix} \end{equation*}$
$$\xi_t = F \xi_{t-1} + v_t $$
from here we can do the same work we did for 1st order difference equations and solve based on a recursive substitution.
I’m not a fan of writing out the matrix tables but it basically is the exact same except arrays vs scalars.
Here’s a simpler one with 2 lags:
$y_t = \phi_1 y_{t-1} \times \phi_2 y_{t-2} + \omega_t$
$\begin{equation*} \begin{bmatrix} y_t\ y_{t-1} \end{bmatrix} = \begin{bmatrix} \phi_1 && \phi_2 \ 1 && 0 \end{bmatrix} \times \begin{bmatrix} y_{t-1}\ y_{t-2} \end{bmatrix} \end{equation*} + \begin{bmatrix} \omega_t\ 0 \end{bmatrix}$
the second identity is $y_{t-1} = y_{t-1}$.
some schnazy matrix algebra and we get the dynamic Multipliers $$ \frac {\delta t_{t+j}} {\delta \omega_t} = f_{11}^{(1)}$$
///////////////////////
BTW … a Fibonacci number is a 2nd order difference equation:
$$x_n = x_{n-1} + x_{n-2} \text{ , } x_0 = 1, x_1 = 1, n = 2,3 …$$