Introduction

Autoregressive (AR) processes form a fundamental component of time series analysis. These processes are integral to understanding the behavior of a variable over time based on its past values. An AR process involves predicting a current value of the variable using a linear combination of its past values, where the coefficients are determined by the model. In this chapter, we delve into the essence of AR processes, their properties, and their practical implications in time series modeling.

Autoregressive (AR) Processes

An autoregressive process of order $p$, denoted as AR(p), is defined as follows:

$$X_t=c+\phi_1 X_{t-1}+\phi_2 X_{t-2}+\ldots+\phi_p X_{t-p}+\epsilon_t$$

Where,

• $X_t$ is the value of the time series at time $t$,
• $c$ is a constant term,
• $\phi_1, \phi_2, \ldots, \phi_p$ are the autoregressive coefficients,
• $X_{t-1}, X_{t-2}, \ldots, X_{t-p}$ are the past values of the time series, and
• $\epsilon_t$ is white noise at time $t$.

Properties of AR Processes

1) Memory: AR processes have memory since the current value depends on its past values. The degree of memory is determined by the order $p$ of the process.

2) Stationarity: The stationarity of an AR process depends on the values of the autoregressive coefficients $\phi_1, \phi_2, \ldots, \phi_p$. For an AR process to be stationary, the absolute values of all roots of the characteristic polynomial must be greater than one.

3) Mean and Variance: The mean of an AR(p) process is constant and equal to $ \frac{c}{(1-\phi_1- \phi_2-\ldots-\phi_p)}$. The variance depends on the white noise term $\epsilon_t$.

4) AutoCorrelation Function: The autocorrelation function (ACF) of an AR(p) process gradually decreases as the lag increases. This indicates a strong dependency on past values.

5) Partial AutoCorrelation Function: The partial autocorrelation function (PACF) shows distinct spikes at lags corresponding to the autoregressive order $p$ and decreases for higher lags.

Example: Consider an AR(2) process defined by $X_t=1+0.5 X_{t-1}+0.3 X_{t-2}+\epsilon_t$. The mean of this process is 2, and its variance depends on the variance of the white noise term $\epsilon_t$. The ACF will show a slow decay, while the PACF will have spikes at lags 1 and 2.