Statistical Inference Using ARMA

Motivation. Given some data over time $X_1,\dots ,X_n$ we want to know if we can use AR/MA to model this. Normally this follows:

1. Prelimiary Analysis
   1. Is the series stationary?
      1. Plot on graph/Numerical tests
      2. If not, can you difference it to get a staionary model? (ARIMA)
   2. Logarithmic?
2. Analysis in the Time Domain: Can we use tools in the Time Series? i.e. Rejecting that it’s just totally random noise
   1. Autocorrelation Function plot (ACF Plot).
   2. Autocorrelation Test
   3. Portmanteau tests (Box-Jenkins, Ljung-Box tests)
3. Model Fitting: Choose $p,q$ parameters, i.e how far to look back. Maybel also $d$.
   1. look at ACF plots to choose
4. Prediction

Analysis in the Time Domain §

let $\{X_t{\}}_{t\in \mathbb{N}}$ a weakly stationary time series. We collect samples $X_1,\dots ,X_n$. Then:

• $\hat{\mu }=\bar{X}=\frac{1}{n}{\sum }_{\forall i}{X}_i$
• $\hat{\gamma }(h)=\frac{1}{n}{\sum }_{t=1}^{n-h}(X_{t+h}-\hat{\mu })(X_t-\hat{\mu })$
• $\hat{\rho }(h)=\frac{\hat{\gamma }(h)}{\hat{\gamma }(0)}$ Motivation. To see if we can use our tools to model the data, we need to see if the time series has some pattern in it, and not just random noise. The hypothesis that it’s just random noise is called: def. SWN hypothesis. This is the Null Hypothesis when we’re checking if our time series has patterns. For given lag $h$, does it have any autocorrelation?

$$H_0:\rho (h)=0$$

If (for many $h$, see below portmanteau tests) we must accept the null, then this time series is not analyzable at all. To develop a formal statistical test with this hypothesis, we use an important property of the estimated autocorrelation function $\hat{\rho }$ that can help us identify things about $\{X_t\}$: thm. Asymptotic normality of causal-weak-stationary Time Series. Let $\{X_t{\}}_{t\in \mathbb{N}}$ be:

1. Causal: $X_t={\sum }_{i=0}^{\infty }{\psi }_i{ϵ}_{t-i}+\mu$
   1. $\mu$: mean of $X_t$, in case mean is not zero
   2. ${\sum }_{i=0}^{\infty }|{\psi }_i|<\infty$
2. Has SWN innovations: $\{{ϵ}_i{\}}_{t\in \mathbb{Z}}\sim \text{SWN}(0,{\sigma }_{ϵ}^2)$
3. Behaves Nicely: $\mathbb{E}({ϵ}_t^4)<\infty$ or ${\sum }_{i=0}^{\infty }i{\psi }_i^2<\infty$ (latter always satisfies by ARMA) then:

$$\sqrt{n}(\hat{\rho }(h)-\rho (h))\stackrel{d}{\to }{N}_h(\vec{0},W)$$

where

• $\hat{\rho }(h)=[\hat{\rho }(1),\dots ,\hat{\rho }(h){]}^{\top }$, same for $\rho (h)$
• $W$ is a matrix with elements: $W_{ij}={\sum }_{k=1}^{\infty }(\rho (k+i)+\rho (k-i)-2\rho (i)\rho (k))(\rho (k+j)+\rho (k-j)-2\rho (j)\rho (k)).$ Example. ==Under Null hypothesis $H_0$== a lag $h$ we have ${\psi }_1=1,{\psi }_{i\ne 1}=0$ and thus:

$$\sqrt{n}(\hat{\vec{\rho }}(h))\stackrel{d}{\to }{N}_h(\vec{0},I_h)$$

This means that under the null hypothesis $\hat{\rho }(h)\sim N\left(0,\frac{1}{n}\right)$. Thus we need to see how much the autocorrelation (per $h$) deviates from normal distribution. With $\alpha =0.05$, $\mathbb{P}(|\widehat{\rho (h)}|>\frac{1.96}{\sqrt{n}})=\alpha =0.05$ and thus this is the rejection region. This is shown as the dotted bands below:

Formally: def. Autocorrelation Test. let time series $\{X_t\}$ with $h$-lag ACF $\rho (h)$. We gather $n$ data points $X_1,\dots ,X_n$. Then the $\gamma$-level test is:

$$\delta :\{\begin{array}{l}{H}_0:\rho (h)=0\text{ iff }|\hat{\rho }(h)|<\frac{{\Phi }^{-1}(\alpha /2)}{\sqrt{n}}\\ H_1:\rho (h)\ne 0\text{ iff }|\hat{\rho }(h)|\ge \frac{{\Phi }^{-1}(\alpha /2)}{\sqrt{n}}\end{array}$$

Prediction §

Motivation. The MSE predictor for a $\text{ARMA}(p,q)$ model is simply: def. MSE Predictor for $\text{ARMA}(1,1)$