We will begin our study of time series by looking at two famous but very simple examples. These will not only give us a feel for the field, but as we will see later on, they will also become integral building blocks to describe more complex time series.
A basic but very important type of time series is known as discrete white noise, or simply white noise. In a white noise time series, the random variables that are generated all have a mean of 0, finite and identical variance σw2, and the random variables at different time steps are uncorrelated with each other. Although some texts do not enforce this requirement, most texts also specify that the variables are also independent and identically distributed (iid) random variables.
The iid property essentially requires that each random variable come from the exact same distribution, such as a normal distribution with a particular mean and standard deviation. The property also requires that two variables from different time steps are not only uncorrelated with each other, but are also independent, which is a stronger requirement.
Note
It is a common misconception in statistics that if two variables are uncorrelated they must also necessarily be independent. While the converse is true, so that two independent variables are automatically uncorrelated, two uncorrelated random variables are not necessarily independent. For example, a random variable X with a normal distribution centered at 0 and a random variable Y, computed as X2, are uncorrelated but clearly not independent of each other.
In this book, we'll include the iid requirement in the definition of the white noise time series. Finally, we'll note that a white noise time series drawn from a normal distribution is known as Gaussian white noise. We can easily generate some Gaussian white noise by sampling from a normal distribution with zero mean using the rnorm() function:
> set.seed(9571) > white_noise <- rnorm(100, mean = 0, sd = 3.0)
The following plot shows our generated Gaussian white noise series:
Suppose we have collected some time series data and want to know if our time series is actually a white noise time series. One way to do this is to plot a correlogram. This is a graph that estimates the ACF function for pairs of time steps that are k steps apart. For different values of k, known as the lag, we should see zero correlation except for the case where k = 0. In this way, we are essentially testing for the uncorrelated property, which is one of the defining characteristics of a white noise series. We can easily do this in R via the acf() function:
> acf(white_noise)
Here is the ACF plot for the white noise series we generated earlier:
The dotted lines show a 95 percent confidence interval within which the value of the ACF function at a particular lag is not considered statistically significant (greater than zero). These values in practice will not be exactly zero due to sampling bias and in fact, we can reasonably expect to see 5 percent of them to be erroneously significant for a correlogram of an actual white noise series.
The correlogram we plotted does indeed match our expectations, whereby only the value at a lag of 0 is statistically significant. To fit our observed time series to a white noise series model, all we need is the variance as this is the only parameter of a white noise series. We can estimate this directly via the var() function, which gives a value very close to what we used to generate the sequence:
> var(white_noise) [1] 9.862699
Another very common type of time series is known as a random walk. This model has been used to describe the motion of pollen particles when placed on the surface of water, known as Brownian Motion, as well as for the price of stocks in the stock market. A time series that is a random walk is a time series in which the differences between successive time points are white noise. We can describe this process using a recurrence relation, that is to say, a relation in which a term in the sequence is defined as a function of terms that occur earlier in the sequence. When using a recurrence relation, we must also specify the initial condition(s) that must hold for the first term(s) in the sequence:
The first equation explains that a general term in the sequence at time t is related to the term in the sequence one time step earlier via the addition of the term et, which is a term from a white noise sequence. The second equation shows the starting condition of the random walk and says that the first term is also the first term of a white noise sequence. Essentially, we can think of a random walk as a sequence that adjusts its current value by a positive or negative amount according to a parallel white noise sequence.
Note
A variant of the random walk is the random walk with drift. This differs from the regular random walk in that we also add a constant term α to every point in the series, and this is known as the drift. Due to this fact, the random walk with drift has a time varying mean of t × α (but the variance is the same as before).
From the definition of a random walk, it should be clear that at time step t, the term in a random walk arises from the sum of t iid variables with 0 mean and identical variance σw2. These are, of course, the terms of a white noise series. Due to this fact, and because of the linearity of expectation and variance, we infer that the mean function of a random walk is 0 for all time points. By contrast, the variance is given by t × σw2. Consequently, in this time series, the variance continually increases over time. To simulate a random walk in R, we essentially need a cumulative sum of a white noise random series. We can do this with R's cumsum() function:
> set.seed(874234242) > random_walk <- cumsum(rnorm(100, mean = 0, sd = 3.0))
Here's the plot of our random walk:
A good way to see if a time series follows a random walk is to compute the successive differences between terms. The time series that is comprised of these differences should be a white noise time series. Consequently, were we to plot a correlogram of the latter, we should see the same kind of pattern characteristic of white noise, namely a spike at lag 0 and no other significant spikes elsewhere.
To compute successive term differences in R, we use the diff() function:
> acf(diff(random_walk))
As expected, this plot looks very similar to the ACF plot of the white noise series we generated earlier in this chapter. Now that we have two examples of time series under our belt, we are ready to tackle an important new concept, stationarity.