Tomorrow's temperature anomaly is about half of today's plus a random shock. A quantity like that takes values on the whole real line, so it cannot be a chain with a finite list of states. But it is Markov: the next value depends only on the current one. Cut the line into a few bands, let each band be a state, and the process becomes an ordinary transition matrix, approximately. This chapter does that for the simplest such process, the AR(1), and compares every answer the chain gives with the exact one.
By the end you can
- Write the conditional distribution of an AR(1) process one step and k steps ahead.
- Derive its stationary distribution from sums of independent normals.
- Turn the process into a finite chain with bands and representative points.
- Compute transition probabilities, stationary probabilities and k-step probabilities of the chain, and compare them with the exact values.
- Explain how a computer solves the large chains that fine bands produce.
A stationary distribution is one that reproduces itself: if Xₙ has it, so does Xₙ₊₁. Try a normal N(m, v) and see what the equation forces.
Choose bands B₁, …, Bₘ that cover the whole line without overlapping, the two outer ones reaching to −∞ and +∞, and a representative point cᵢ inside each band. State i means "the value is in band i", and to move on we pretend the value is exactly cᵢ. Then the next value is N(φcᵢ, σ²), and the transition probability to band j = [lⱼ, uⱼ) is a normal probability, computed with Φ, the standard normal distribution function:
Worked example: φ = 0.5, σ = 1, three bands
Bands (−∞; −1), [−1; 1) and [1; ∞), represented by −2, 0 and 2. From −2 the next value is N(−1, 1); from 0 it is N(0, 1); from 2 it is N(1, 1).
| from ↓ to → | −2 | 0 | 2 |
|---|---|---|---|
| −2 | 0.5000 | 0.4772 | 0.0228 |
| 0 | 0.1587 | 0.6827 | 0.1587 |
| 2 | 0.0228 | 0.4772 | 0.5000 |
Three bands are crude. Bands of width 1/11 from −6 to 6 give 133 states, and the chain then agrees with the exact answers to three or four decimals. Nobody solves 133 equations by hand, but the computer does nothing new. It follows one of two routes. (1) Elimination: write πP = π as (P − I)ᵀπ = 0, replace one of the equations (they are redundant by one) by π₁ + … + πₘ = 1, and eliminate unknowns exactly as you would for three. (2) Iteration: start from any distribution and multiply by P again and again, π ← πP, until it stops changing; for a chain like this one it converges to π (the convergence result of the Markov-chains chapter). k-step probabilities need only the starting row multiplied by P k times, never Pᵏ itself. Any statistics package, R included, uses one of these two routes. The figure below does the same in your browser.
Blue bars: π of the chain divided by the band width, so they can be compared with a density; the two outer bands, which reach to ±∞, are drawn as hatched blocks of their probability. Amber curve: the exact stationary density. Shrink w and widen the range and the bars follow the curve; with w = 1/11 and points up to ±6 the chain has 133 states, solved here in a moment. The second line of numbers answers a k-step question both ways: the chain multiplies the starting row by P k times; the exact answer uses X(n+k) | x₀ ~ N(φᵏx₀, σ²(1 − φ²ᵏ)/(1 − φ²)).
| Step | What to do |
|---|---|
| 1 | List the bands and their points; make the two outer bands reach −∞ and +∞. |
| 2 | For each point cᵢ: the next value is N(φcᵢ, σ²). Standardise every band limit: z = (limit − φcᵢ)/σ. |
| 3 | P_ij = Φ(z upper) − Φ(z lower). Check that each row adds up to 1; use symmetry to save work. |
| 4 | Solve πP = π with Σπ = 1 (by hand for 3 bands; by elimination or iteration for many). |
| 5 | Exact comparison: band probabilities under N(0, σ²/(1 − φ²)). |
| 6 | k-step questions: row of the starting band times P, k times, then add the bands asked for; exact: N(φᵏx, σ²(1 − φ²ᵏ)/(1 − φ²)). |
A round of 10 questions
Answer as many as you can. Mistakes come back at the end for a review round.
Every exercise is made on the spot, so a new round is never the same. The computer picks the answer first and builds the question around it, working with exact fractions, so the answer key is always right. Answers are read exactly: 0.75, 3/4 and 6/8 count as the same number (where lowest terms are asked for, 6/8 just gets a reminder). A wrong answer shows the worked solution line by line, in the order the lesson does it; the new number in each line is coloured.
| Wrong | Right | Why |
|---|---|---|
| next value ~ N(cᵢ, σ²) | N(φcᵢ, σ²) | the bell is centred on φ times the current value |
| outer bands cut at the last point | outer bands to ±∞ | otherwise the rows do not add up to 1 |
| P_ij ≈ density at cⱼ | Φ(upper) − Φ(lower) | a transition probability is an area, not a height |
| compare with N(0, σ²) | N(0, σ²/(1 − φ²)) | the stationary spread includes all the shocks still remembered |
| two-step variance 2σ² | σ²(1 + φ²) | the older shock has been multiplied by φ once |
| standardise with σ² | divide by σ (the standard deviation) | z = (x − mean)/standard deviation |
Key ideas
- AR(1): Xₙ₊₁ = φXₙ + ε, so Xₙ₊₁ | Xₙ = x ~ N(φx, σ²); it is Markov.
- Stationary distribution (|φ| < 1): N(0, σ²/(1 − φ²)), from v = φ²v + σ².
- k steps ahead: N(φᵏx, σ²(1 − φ²ᵏ)/(1 − φ²)).
- Bands + representative points: P_ij = Φ((uⱼ − φcᵢ)/σ) − Φ((lⱼ − φcᵢ)/σ), outer bands to ±∞.
- The chain's π and Pᵏ approximate the exact answers; narrower bands approximate them better.
- Large chains are solved by elimination with Σπ = 1 or by iterating π ← πP.