Time Series

def. Time series defined on a probability space $(\Omega ,\{{\mathcal{F}}_t{\}}_{t\in \mathbb{Z}},\mathbb{P})$ with filtration indexed by integers is $\{X_t{\}}_{t\in \mathbb{Z}}$ with adapted to the filtration.

Moments of Time Series §

def. Moments of time series $\{X_t\}$ are:

$$\begin{array}{rl}\text{Mean}:{\mu }_t& =\mathbb{E}[X_t]\\ \text{Autocovariance}:{\gamma }_{s,t}& =\mathbb{E}[(X_t-{\mu }_t)(X_s-{\mu }_s)]\end{array}$$

def. Strong Stationarity. $\{X_t\}$ is strongly stationary iff $\forall (1\dots n),\forall k$

$$(X_{t_1},\dots ,X_{t_n})\stackrel{d}{=}(X_{t_1+k},\dots ,X_{n+k})$$

• ! This doesn’t mean that moments exist i.e. $\mathbb{E}[X_t]=\infty$ but also strongly stationary.
• The reason why you don’t write $X_{t_1}\stackrel{d}{=}{X}_{t_2}\stackrel{d}{=}\dots$ Intution. If inspected for any time period of length $n$, its same as any other period of length $n$. def. Weak Stationarity (=Covariance Sationary). $\{X_t\}$ is weakly stationary iff

$$\begin{array}{rlr}\forall t,& {\mu }_t=\mathbb{E}[X_t]& \text{mean is same}\\ \forall t,s,k,& {\gamma }_{t,s}={\gamma }_{t+k,s+k}& \text{covariance is same}\end{array}$$

• ! This means that moments do exist, i.e. always $\mathbb{E}[X_t],\mathbb{E}[X_t^2]<\infty$ Intution. If inspected at any point, the distribution have the same first and second moments.
• For a weakly stationary process, the covariance $\gamma (s,t)$ only depends on the temporal separation $|s-t|=:$ lag. So we write $\gamma (t,t+k)=\gamma (0,k)=\gamma (k)$. def. Autocorrelation Function (ACF). Weakly stationary process $\{X_t\}$ with mean $\mu$ and variance $\gamma (\cdot )$ is

$$\rho (h):=\rho (X_h,X_0)=\frac{\gamma (h)}{\gamma (0)}$$

Motivation. Seems like it’s simply the standardized version of covariance. But ACF is very important because it fully characterizes the dependence structure of the process. ¶we can use the sample autocorrelations, compare with the theoretical ACF and see which model to use.

Simple Processes §

def. White Noise Process. (WN). $\{{ϵ}_t\}$ is a $\text{WN}(\mu ,{\sigma }^2)$ process iff

1. ${ϵ}_t$ has central moments $\mu ,{\sigma }^2$
   1. i.e. Weakly stationary
2. ! AND $\forall h\ne 0,\rho (h)=0$ i.e. no two time points are correlated def. Strict White Noise Process (SWN). $\{{ϵ}_t\}$ is a SWN process iff:
3. ${ϵ}_i$ are all $iid$
4. $\mathbb{E}[X_t]=0$, $\text{Var}[X_t]<\infty$ def. Martingale Difference Process. $\{X_t\}$ adapted to filtration $\{{\mathcal{F}}_t\}$is a martingale difference process if:

$$\mathbb{E}[X_t\mid {\mathcal{F}}_s]=0$$

Compare this to a normal martingale process. It’s called martingale difference because it serves as “difference” process, i.e. there exists a martingale process $\{M_t\}$ st:

$$X_t=M_{t+1}-M_t$$

Causal Process. §

Motivation. You may naiively think that $\{X_t\}$ being adapted to filtration $\{{\mathcal{F}}_t\}$ is enough to make it “causal.” And technically that is true. But consider the case of $\text{AR(1)}$. let $\{X_t\}$ be $\text{AR}(1)$ with coefficient $\varphi =2$

$$X_t=2X_{t-1}+{ϵ}_t$$

Expanding this recursively we see that this is explosive:

$$X_t=\sum _{i=0}^{\infty }2\cdot {ϵ}_{t-i}\stackrel{n\to \infty }{\to }\infty$$

Explosive processes like these have no real applicatbility in statistics. So we consider weakly stationary processes only. def. Causal Process. $\{X_t\}$ is a causal process if

1. It is weakly stationary
2. It is adapted to filtration $\{{\mathcal{F}}_t\}$. This is equivalent to the following definition: $\{X_t\}$ can be written as an infinite series of white noise $\{{ϵ}_t\}\sim \text{WN}(0,{\sigma }_{ϵ}^2)$:

$$X_t={\psi }_1{ϵ}_{t-1}+{\psi }_2{ϵ}_{t-2}+\cdots =\sum _{i=0}^{\infty }{\psi }_i{ϵ}_{t-i}$$

where

• ! we must have ${\sum }_{i=0}^{\infty }|{\psi }_i|<\infty$, else it is not stationary.
• Naturally, causal process can be expressed into an $\text{MA}(\infty )$ process that isn’t explosive
• AutoCov and ACF is:

$$\begin{array}{rl}\gamma (h)& ={\sigma }_{ϵ}^2\sum _{i=0}^{\infty }{\psi }_i{\psi }_{i+|h|}\\ \rho (h)& =\frac{{\sum }_{i=0}^{\infty }{\psi }_i{\psi }_{i+|h|}}{{\sum }_{i=0}^{\infty }{\psi }_i^2}\end{array}$$

Invertible Process. §

Motivation. This is also a housekeeping-style definition so that we can construct the current shock based on past data. def. Invertible Process. $\{X_t\}$ is an invertible process it can be expressed in the form:

$${ϵ}_t=\sum _{i=0}^{\infty }{\pi }_i{X}_{t-i}$$

• Naturally a $\text{AR}(\infty )$ process