Chapter 5: Tests for Differencing Time Series

Introduction

Stationarity

Unit Roots

Dickey-Fuller Tests for Unit Roots

Simple Applications of the Dickey-Fuller Test

Augmented Dickey-Fuller Tests for Milk Production

KPSS Unit Root Tests

An Application of the KPSS Unit Root Test

Seasonal Differencing

Conclusion

Introduction

This chapter describes how to test for unit roots in order to judge whether a first-order differencing is necessary to obtain stationarity. In time series theory, stationarity is a clear advantage. Most series with a trending behavior are transformed into stationarity when differences are made. From an intuitive viewpoint, this method also clearly points toward real relationships because changes in the series are modeled by changes in other series.

It is possible to test whether unit roots are present in an observed time series. In this chapter, two tests are discussed, and they are demonstrated with the use of PROC AUTOREG.

Stationarity

In probability theory, a univariate time series is said to be stationary if certain explicit assumptions are met. In this book, it suffices to state that a series is stationary if all relationships between the values are unchanged when all time indices are shifted by the same number of time units. In particular, the mean value and the variance must be constant. Moreover, the autocorrelations, which are the correlations ρ_k = corr(x_t, x_t−k), between the values of the series at different points in time, must be constant. This means that the autocorrelations do not depend on the time index t, but depend only on the time span k.

The concept of stationarity forms a useful base for probabilistic theorems about forecasting and modeling time series. Stationarity is also an intuitive concept when you forecast a time series, because stationarity means that structures observed in the past will also be present in the future. This is a fundamental assumption that underlies prediction methods.

The simplest example of a stationary time series is a so-called white noise sequence ε_t, where all values are independent, identically distributed, stochastic variables. The error terms in a regression model for time series data are usually assumed to form a white noise series, often with a further assumption of normality. A popular example of a stationary time series model is the first-order autoregressive, AR(1) model x_t − φ₁x_t−1 = ε_t, which was applied in Chapters 2 and 3. A first-order autoregressive time series is stationary if |φ₁| < 1.

Some time series are obviously stationary, as when, say, a physical theory states that a time series is stable and the observed series behaves rather constant. But other time series are obviously not stationary as in, say, economics, where inflation and economic growth lead to steadily increasing levels. For many time series, a quick glance at a graph of the series can reveal whether the series is stationary.

But in other situations, the presence of stationarity is less obvious. To help in such cases, various test procedures have been suggested and implemented in SAS. The hypothesis of stationarity is not, however, a simple hypothesis: Many different time series models are formulated for stationary time series. In this chapter, two data examples are given to illustrate how to apply these tests.

Unit Roots

A nonstationary series for which the series of first differences x_t − x_t−1 is stationary is said to have a unit root. This notation is based on the first-order autoregressive model x_t − φ₁x_t−1 = ε_t being considered as a polynomial 1 − φ₁B. Here B denotes the backward shift operator, Bx_t = x_t−1. The root in this polynomial is 1/φ₁, which is numerically larger than 1 when the series is stationary. The value φ₁= 1 is then the boundary value, where the root of the autoregressive polynomial is unity. Often, the series of first differences x_t − x_t−1 is denoted Δx_t  = x_t − x_t−1.

One example of a time series model with a unit root is a random walk, which is defined as  x_t − x_t−1 = ε_t , where ε_t is assumed to be white noise. This model is often applied to stock market data. It is intuitively correct to consider it, for example, for daily changes in the rate of a stock. This is because the series of daily changes is the series of real interest rather than the quoted rate itself. The random walk model is rather boring from a time series analysis point of view, because there is nothing to model, and the last observed value is the best prediction for future observations. Instead, the interest turns to the variance of the series ε_t, which is often non-constant and often modeled by means of GARCH models, which are the focus in Chapter 15.

If the series possesses a linear trend,

$$x_t\text{=α}+\text{β}t+{\zeta }_t$$

then the slope parameter β is simply the mean value of the stationary series:

$$\Delta x_t=x_t-x_{t\text{-1}}=\text{β}+{\text{ζ}}_t-{\text{ζ}}_{t-1}$$

If the series Δζ_t  = ζ_t − ζ_t−1 has reasonable time series properties like stationarity, the parameter β is easy to estimate. In the resulting model, the observed series x_t has the form of a linear trend, which is overlaid by a time series model.

Some time series, like the Gross National Product, are not stationary. But the series of first-order differences, x_t − x_t−1, the growth, could be assumed to be stationary and rather constant in the long term. Good and bad times in the economy could then be seen as autocorrelation in the series, which perhaps could be modeled as a first-order autoregressive model or by more refined models presented later in this book.

Differencing time series is a crucial part of the theory of cointegration and error correction models. These rather advanced models are specific parameterizations to model the data-generating process underlying multivariate time series. Such models are studied in Chapters 13 and 14.

Dickey-Fuller Tests for Unit Roots

The simplest situation with a unit root is a first-order autoregressive model, x_t − φ₁x_t−1 = ε_t, where the autoregressive parameter φ₁ equals +1. At first sight, it seems a simple testing problem. But because the value +1 is on the boundary and not included in parameter space, the test is not that easy.

In the most popular test, the Dickey-Fuller test, the problem is reformulated as shown here:

$$\Delta x_t=\text{α}{x}_{t-1}+{\text{ε}}_t$$

where the parameter α is α = 1 − φ₁. The value α = 0 corresponds to the situation φ₁ = 1.

The hypothesis φ₁ = 1 is tested by the test statistic, usually called τ:

$\text{τ}=\frac{\widehat{\alpha }}{\sqrt{\mathrm{var}\left(\widehat{\alpha }\right)}}$

where the estimate of the parameter α is calculated by ordinary least squares (OLS) estimation. The denominator is the usual standard deviation for the OLS estimation. The calculation of this test statistic can easily be performed by a simple application of PROC REG. The problem is that the statistic under the null hypothesis α = 0 is not a Student-t distribution as in usual regression. Instead, the distribution has the form of an integral of a Wiener process. This result relies on advanced probabilistic arguments. Most time series statisticians would prefer not to pursue such results and would rather apply the output tables.

A similar test statistic is the ρ-test defined by the following formula:

$\text{ρ}=T\widehat{\alpha }$

It must be stressed that the Dickey-Fuller test has a unit root as its null hypothesis, which means that this hypothesis is accepted unless the data provides sufficient evidence to reject it. So, in practice, a unit root is often accepted in situations in which it is meaningless (as in, for example, a physical model). For an alternative with stationarity as the null hypothesis, see the section, “KPSS Unit Root Tests.”

Dickey-Fuller testing is easily extended by including lags (called augmented lags) in the basic situation so that the model becomes as follows:

$$\Delta x_t=\text{α}{x}_{t-1}+{\displaystyle \sum _{j=1}^p{\text{α}}_j\Delta x_{t-j}}+{\text{ε}}_t$$

The extra terms correspond to the situation that the series of first-order differences form an autoregressive model of order p. (The differences are just assumed to be independent white noise in the first formulation of the Dickey-Fuller test.) The order, p, of the model is often found by an automatic fitting procedure that compares the fit of autoregressive models of different orders like the method described in Chapter 8.

In this situation, a unit root is still tested by testing α = 0, using the same τ-test statistic. The test is then referred to as the Augmented Dickey-Fuller (ADF) test.

The ρ-test statistic has to be corrected for the lags, Δx_t−j so that it becomes:

$$\text{ρ}=\frac{T\widehat{\alpha }}{1-{\widehat{\alpha }}_1-\mathrm{..}-{\widehat{\alpha }}_p}$$

Moreover, a constant term in the equation and even a trend could be added to the basic form of the Dickey-Fuller test statistic. In this way, the acceptance of a unit root, which is only because of a linear trend, is avoided. Inclusion of a linear trend or a constant term changes the distribution of the test statistics.

The test statistic, in general, comes from testing the hypothesis α = 0 in the following relation:

$$\Delta x_t=\text{α}{x}_{t-1}+\text{μ}+\text{β}t+{\displaystyle \sum _{j=1}^p{\text{α}}_j\Delta x_{t-j}}+{\text{ε}}_t$$

This extension of the model by a constant term and a linear trend changes the distribution, but the SAS procedures provides the relevant p-values.

Simple Applications of the Dickey-Fuller Test

Program 5.1 gives the Dickey-Fuller test for the number of cows series, by means of the option STATIONARITY=(ADF).

Program 5.1: Calculating the Dickey-Fuller Test for the Number of Cows Series

The resulting table, Output 5.1, gives the test statistic in three situations. The first row is the zero mean model, which is the simplest regression, with no constant term and no trend included. The regression, including a constant value, μ, is in the second row. The further inclusion of a linear trend is in the last row. Both statistics τ and ρ are printed in the table. The conclusion is that the hypothesis is accepted in all situations and that a unit root is present. The plot of the series of differences (Figures 4.1 and 4.2) similarly shows that the differenced series seem stationary.

Output 5.1: Results for Dickey-Fuller Unit Root Test for the Number of Cows

The augmented Dickey-Fuller test using 5 lags is printed by extending the option by the number 5 to the form STATIONARITY=(ADF=5). The number 5 is chosen because five autoregressive terms can be relevant in a quarterly series. The exact number is not important; here, the number 5 is chosen just because it’s a bit larger than 4. The test conclusions are unchanged; Output 5.2.

Output 5.2: Results for the Augmented Dickey-Fuller Unit Root Test for the Number of Cows

Augmented Dickey-Fuller Tests for Milk Production

For milk production, a clear seasonal pattern is present. In this situation, the assumptions underlying the Dickey-Fuller test are not directly met. Program 5.2 performs the tests. Here, 9 augmented lags are applied because the quarterly structure is modeled by autoregressive terms at lags up to lag 9 (that is, for more than two years).

Program 5.2: Calculation of the Dickey-Fuller Test for the Milk Production Series

The result is that a unit root is accepted if the series is assumed to have zero mean or a single mean (a constant parameter μ different from zero in the formula).  This is, however, clearly not the case in the plot of the series (Figure 2.1), where an upward trend is obvious for the production series. Under the assumption of a linear trend, the hypothesis of a unit root is accepted by the τ statistic, but it is clearly rejected (p < .0001) by the ρ statistic. A rejection means that if a linear trend is included in a model for this series, a unit root is not present in the error term.

Output 5.3: Results for Dickey-Fuller Unit Root Test for the Number of Milk Production

In the present situation, it is a good idea to model both series at the same level of trending and differencing. Even if the trending behavior could be modeled by a linear trend using no differencing, the series might also be modeled by a differencing using no linear trend. For the original series, the trend parameter, β, turns into the mean value in the series of differences. This is seen in Figure 4.1 where the level of the differenced series seems constant.

In the original Box-Jenkins (1976) procedure, differencing is often applied for trending series. The argument is that even if a trend exists, the trend is of no importance for short time spans like a year. This is also clear from Figure 4.1, where the mean value is seen to be close to zero. Time series models like the Box-Jenkins (1976) ARIMA models describe only short-time behavior of the series.

KPSS Unit Root Tests

In this section, the choice of null and alternative hypotheses in the previous sections are reverted. In many practical situations, it is most natural to consider stationarity as the null hypothesis. Stationarity is often a type of steady state in models—for example, in econometric models. Often, a unit root signals that the system is out of balance, meaning that the underlying theory is wrong. The choice of null hypothesis also has the effect that the null is accepted in all situations and rejected only in cases in which the data tells a different story with high strength.

However, it is not easy to specify a null hypothesis of stationarity, because stationarity is a composite hypothesis that consists of many models. Often stationarity means that an autoregressive model, a moving average model, or some combined Autoregressive Moving Average (ARMA) model could be fitted to the series. But the concept of stationarity is broader than this. To operate the null hypothesis, Kwiatkowski, Phillips, Schmidt, and Shin (KPSS), in a sequence of papers, introduced an algorithm that has proven to work well. (For references, see the SAS online Help.) The test is called the KPSS unit root test.

The test relies on the idea that a stationary residual series ε_t that is not necessarily non-autocorrelated, is complemented by an additional series that is a random walk, η_t = η_t−1 + ξ_t. The hypothesis of no unit root is formulated as the hypothesis that the variance of ξ_t, is 0, which means that the random walk reduces to a constant. The actual test for this hypothesis is a Lagrange Multiplier test, which relies on the autocorrelation structure of the stationary part of the residuals. The test statistic is a rather complicated expression, and the distribution under the null is again an expression that includes stochastic integrals. As with the Dickey-Fuller test, the distribution depends on whether a constant term or a linear trend is included in the model.

An Application of the KPSS Unit Root Test

Program 5.3 gives the code for performing the KPSS test for the number of cows. No seasonal factors are included because the seasonality, see Figure 2.1, is of only minor importance for series of the number of cows.

Program 5.3: Coding for the KPSS Unit Root Test in PROC AUTOREG

The hypothesis of stationarity is accepted by the KPSS unit root test. (See Output 5.3.) This conclusion is the opposite of the result for the Dickey-Fuller test, which accepted the hypothesis of a unit root. (See Output 5.1.) The test results are then concluded to be ambiguous. In statistical terms, this is because two different null hypotheses are applied for the two tests, and the power of the tests is rather small. These conditions lead to the conclusion that neither of the two null hypotheses are rejected. Your decision as to whether to use a unit root must be based on other information or simply on which of the two model features (a unit root or not) seem most suitable for the final model.

Output 5.4: Results of the KPSS Unit Root Test in PROC AUTOREG

In this situation, strong autocorrelation is present. (See the autocorrelation plot, Figure 5.1.) No independent variable is specified in Program 5.3. And the plot is simply the autocorrelations for the original series for the number of cows even if the title uses the term “Residuals.” This autocorrelation in itself tells that the series is probably a stationary first-order autoregressive process.

Figure 5.1: Autocorrelations for the Series of Number of Cows

Seasonal Differencing

In the commonly used Box-Jenkins framework, seasonal differencing is an often applied remedy to transform a series into stationarity. A seasonal difference for quarterly data is defined by z_t = x_t − x_t−4. For the autoregressive polynomial, this corresponds to the polynomial 1 − B⁴, using the backward shift operator B. This polynomial has four roots that are equally distributed on the unit circle in the complex plane. The interesting question is then whether this fourth-order difference forms a stationary time series.

If the series has a constant additive seasonal structure, the series of fourth-order differences needs no seasonal dummies because they disappear with the differencing. A fixed seasonal structure is accounted for by seasonal differencing, but a fourth-order difference can be the right tool even in other cases. This might be the case where seasonality changes in a way that lets the changes in seasonality take the form of a random walk.

One problem in models where a seasonal difference is implied is that changes in the series appear only as happening over a full year. If an increment in milk production is concentrated in just one particular quarter, all yearly differences, including this quarter, will be positive. For an increasing level of a time series, the observation of each quarter will be larger than the observation in the same quarter the year before.

One way to see in which quarter something happened to the series is to compare the yearly differences. In modeling terms, this means that if events occur over shorter spans than a year, first-order differences must be implied to the series of fourth-order differences. The basis is that a double differencing is applied:

$\left(1-B\right)\left(1-B^4\right)=1-B-B^4+B^5$

In this way, many seasonal time series with a trend are transformed into stationarity. This method is often applied in the Box-Jenkins framework for seasonal time series.

If a fourth-order difference is applied, the first four observations are used only to define the difference for the next observations. Four observations are left out. This means that the observed fourth-order differences span from 5, . . .  T, and the number of observations is T − 4. When differencing is applied then an intercept term is usually equal to 0. The alternative to seasonal differencing is to apply 3 seasonal dummies and an intercept. This also reduces the total number of degrees of freedom by 4. So from an information theoretical point of view there is no difference.

The Dickey-Fuller test for a unit root is generalized to the hypothesis of a seasonal root. This test is, however, not implemented in PROC AUTOREG, but it is available using PROC ARIMA or PROC VARMAX.

Conclusion

In this chapter, the concept of differencing a time series is continued from Chapter 4, but it now is seen from a more theoretical point of view.

In the Box-Jenkins (1976) method, differencing is seen as a method for transforming a time series into stationarity in cases in which the series has shifting levels that possibly could have the form of what is usually denoted as a trend. In this context, the decision to apply differencing is made mainly by looking at the series.

In cointegration, unit roots are the base for rather advanced econometric modeling of multidimensional time series (see Chapters 13 and 14), In this context, the analyst has a clear need to test for the presence of unit roots.

In this chapter, the Dickey-Fuller test and the KPSS test are described, and these two tests are applied by PROC AUTOREG. It is also possible to calculate Dickey-Fuller tests with PROC VARMAX, which is the main procedure applied in this book. This alternative is demonstrated in the following chapters, beginning with Chapter 7, which introduces PROC VARMAX.