Many models that we come across involve observing a process of some sort over a period of time in order to learn to predict how that process will behave in the future. As we are dealing with a process that generates observations indexed by time, we refer to these models as time series models. Classic examples of time series are stock market indexes, volume of sales of a company's product over time, and changing weather attributes such as temperature and rainfall during the year.
In this chapter, we will focus on univariate time series, that is to say, time series that involve monitoring how a single variable fluctuates over time. To do this, we begin with some basic tools for describing time series, followed by an overview of a number of fundamental examples. It turns out that there is a wide variety of different approaches to modeling time series; in this chapter, we will focus primarily on ARIMA models, but we will also provide pointers on a few alternatives.
A time series is just a sequence of random variables, Y1, Y2, …, YT, indexed by an evenly spaced sequence of points in time. Time series are ubiquitous in everyday life; we can observe the total amount of rainfall in millimeters over yearly periods for consecutive years, the average daytime temperature over consecutive days, the price of a particular share in the stock market at the close of every day of trading, or the total number of patients in a doctor's waiting room every half hour. As we can see, examples abound.
To analyze time series data, we use the concept of a stochastic process, which is just a sequence of random variables that are generated via an underlying mechanism that is stochastic or random, as opposed to deterministic. From the perspective of the predictive modeler, our goal is to study time series in order to build a model that best describes the behavior of a finite set of samples that we have obtained, in order for us to predict how the time series will behave in the future.
For example, if we have a vested interest in foreign currency exchange rates, we might want to study how the exchange rate between the Euro and the British Pound changed over a period of time in the past, in order for us to predict what the exchange rate might be in the near future. This could help us make a decision as to whether we should convert the Euros we have now into Pounds, or wait to convert them later if we expect the exchange rate to work in our favor in the near future.
At first glance, we might think that an approach to analyzing a time series would be to obtain the probability distribution of every variable in the sequence. In actuality, our time sequence can be quite large and this approach is impractical and unrealistic. Instead, we will begin by summarizing the main properties of the joint probability distribution of our random variables using the mean, covariance, and correlation functions.
From basic statistics, we are already familiar with the ideas of mean, variance, covariance, and correlation when it comes to two random variables. We can take these ideas and apply them in the context of time series by treating each point in time as a random variable. The mean function of a time series is the expected value of the time series at a particular time index, t:
In the general case, the mean of a time series at a particular time index t1 is not the same as the mean of that time series at a different time index t2. The autocovariance function and autocorrelation function are two important functions that measure the linear dependence between the random variables that make up the time series at different time points. The autocorrelation function is commonly abbreviated to the ACF function. For two time indexes, t1 and t2, we compute the autocovariance function as:
When the two time indexes are the same, the autocovariance function is just the variance:
The variance and autocovariance functions are measured in squared units of the original time series output units. Additionally, the autocovariance is a symmetric function, in that the same result is obtained if we switch the two time indexes in the computation. The ACF function is also symmetric, but it is unitless and its absolute value is bounded by the value 1. When the autocorrelation between two time indexes is 1 or -1, there is a perfect linear dependence or correlation, whereas when the value is 0, then the two time indexes are said to be uncorrelated. The ACF function is computed as follows:
It should be easy to spot from the previous equation that the ACF when computed on two identical time indexes just produces the value 1 since the covariance in the numerator just reduces to the variance. This result makes sense as any random variable is trivially perfectly correlated with itself.