Note 69. Fitting AR Models to Stochastic Signals: The Yule-Walker Method

This note describes the Yule-Walker method for finding the parameters needed to fit an autoregressive model to a finite sequence of samples obtained from a stochastic signal. This method is conceptually straightforward; it is based on the simple idea of substituting an estimated autocorrelation sequence (ACS) for the true ACS in the Yule-Walker equations that are presented in Note 68. Other methods for fitting an AR model to a finite sequence of signal samples are presented in Notes 72 and 73.

The Yule-Walker method is a technique for fitting an autoregressive model to a stochastic signal that is assumed to be auto regressive, but where knowledge about the signal is limited to a sequence of N samples, x[0] through x[N –1]. The Yule-Walker equations for an AR process are given as

69.1

where

The Yule-Walker method is based on using estimated values for the autocorrelation sequence (ACS) to populate the R matrix in Eq. (69.1). Assuming that x[n] is known only for N values x[0] through x[N –1], the estimated ACS values, r_x[–p+1] through r_x[p], can be obtained as

69.2

Because the matrix R is Toeplitz, the Levinson recursion can be used, as shown in Recipe 69.1, to solve for the coefficients a_p[1] through a_p[p].

69.1. About Recipe 69.1

Equation (69.3) is the biased estimate of the ACS for finite N. The unbiased estimate is the same, except for a normalizing factor of 1/(N – k) that replaces the factor of 1/N in Eq. (69.3). Normally, it is preferable to use unbiased rather than biased estimators, but for values of k that approach the value of N, the unbiased ACS estimator can produce results where the autocorelation estimate at lag zero is smaller than the estimate at one or more non-zero lags. This result can sometimes lead to matrix equations that cannot be solved [4]. Therefore, the biased ACS estimate is almost always the one used in the Yule-Walker method.

Along the way to producing the coefficients, a_p[i], for an AR(p) model, the Levinson recursion generates coefficients for all of the lesser-order models, AR(1) through AR(p –1).

References

1. M. H. Hayes, Statistical Digital Signal Processing and Modeling, Wiley, 1966.

2. N. Levinson, “The Wiener RMS (Root Mean Square) Error Criterion in Filter Design and Prediction,” J. Math Phys., vol. 25, 1947, pp. 261–278.

3. J. Durbin, “The Fitting of Time Series Models,” Rev. Inst. Int. Statist., vol. 28, 1960, pp. 233–243.

4. S. L. Marple, Digital Spectral Analysis with Applications, Prentice Hall, 1987.