9.3.1 The smoothed spectrum matrix

Given a zero‐mean m‐dimensional time series, Z₁, Z₂, …, and Z_n, its Fourier transform at the Fourier frequencies ω_p = 2πp/n, − [(n − 1)/2] ≤ p ≤ [n/2], is

(9.44)

Then, the m × m sample spectrum matrix, which is also known as periodogram matrix, is simply the extension of Eqs. (9.9)–(9.11). Thus,

(9.45)

where

and

When Z_t is a multivariate Gaussian process with mean vector 0 and variance–covariance matrix Σ, has a distribution related to the sample variance–covariance matrix that is known as Wishart distribution with n degrees of freedom, which is the multivariate analog of the Chi‐square distribution. We refer readers to Goodman (1963), Hannan (1970), and Brillinger (2002) for further discussion of the properties of the periodogram and Wishart distribution.

Similar to the univariate extension of the sample spectral density discussed in Section 9.1, the sample spectrum matrix or periodogram matrix is also a poor estimate. So, we replace it by the following smoothed spectrum matrix

(9.46)

where

(9.47)

W_i(ω) is a smoothing function, also known as kernel or spectral window, and M_i is the bandwidth of the spectral window, and

(9.48)

where W_i,j(ω) is a spectral window, and M_i,j is the corresponding bandwidth. Similar to the extension of the univariate smoothed spectrum, the smoothed spectrum matrix can also be approximated by the Wishart distribution.

Once f_i,i(ω) and f_i,j(ω)are estimated, we can estimate the co‐spectrum, c_i,j(ω), the quadrature spectrum, q_i,j(ω), the cross‐amplitude spectrum, α_i,j(ω), phase spectrum, ϕ_i,j(ω), the gain function, G_i,j(ω), and the squared coherency function,

Note that the spectrum matrix is the Fourier transform of the covariance function, Γ(k) = [γ_i,j(k)], and the sample spectrum matrix is

(9.49)

Instead of spectrum smoothing, we can also apply the smoothing function to the sample covariance matrices, that is,

(9.50)

where

(9.51)

is the sample autocovariance for the Z_i,t series, W_i(k) is a suitable lag window, and M_i is the truncation point, and

(9.52)

is the sample cross‐covariance function between Z_i,t and Z_j,t, W_i,j(k) is a suitable lag window, and M_i,j is the corresponding truncation point. In this case, the estimations of the co‐spectrum, the quadrature spectrum, the cross‐amplitude spectrum, the phase spectrum, the gain function, and the squared coherency function are given by

(9.53)

(9.54)

(9.55)

(9.56)

(9.57)

and

(9.58)

For commonly used spectral or lag windows and their properties, we refer readers to Priestley (1981), Brillinger (2002), and Wei (2006).

Some important remarks are in order here.

1. The smoothed lag windows and the associated truncation points used in estimating f_i,i(ω) and f_i,j(ω) are not necessarily the same for all i and j. Even the same lag window is used, the associated truncation points can be different. However, the use of the same lag window and truncation point for all estimates will in general make the study of sampling properties of the estimates simpler.
2. In estimating the cross‐spectrum, the lag window places heavier weight on the sample cross‐covariances around the zero lag. If one series leads another and the cross‐covariance does not peak at zero lag, then the estimated cross‐spectrum will be biased. This bias is especially severe for the estimated square coherency. Because the square coherency is invariant under the linear transformation, to reduce the bias, one can properly align two series by shifting one of the series by a time lag τ so that the peak in the cross‐spectrum function of the aligned series occurs at the zero lag. In practice, the time lag τ can be estimated by the lag corresponding to the maximum cross‐correlation.
3. The estimates of the cross‐amplitude, phase, and gain functions are not reliable when the square coherency is small.
4. It should be noted that when we write the spectral density as , it implies that the range of the frequencies is from −π to π. Given a time series of length n, due to symmetry, the Fourier frequencies used in the sample spectrum will be ω_j = 2πj/n, with j = 0, 1, …, [n/2]. However, if one writes the spectral density as it implies that the range of the frequencies will be −1/2 to 1/2. In this case, the frequencies used in the sample spectrum will be ω_j = j/n, with j = 0, 1, …, [n/2]. In the following discussions, we may use either one depending on what is used in software and the referenced papers.

9.3.2 Multitaper smoothing

Developed by Thompson (1982) for univariate processes and extended by Walden (2000) for multivariate processes, multitaper smoothing is another useful way to estimate power spectrum density that balances the bias and variance of nonparametric spectral estimation. The multitaper reduces estimation bias by averaging modified periodograms obtained using a family of mutually orthogonal tapers from the same sample data. Let h_j(t) for t = 1, …, n and j = 1, …, n, be n orthonormal tapers such that

(9.59)

From Eq. (9.45), we note that

(9.60)

where is the discrete Fourier transform of Z_t. The multitaper power spectral estimator at frequency ω is

(9.61)

where K is chosen through the method shown below and is the tapered Fourier transform such that

(9.62)

The multitaper spectral estimator is asymptotically unbiased and is consistent when the number of tapers K increase with the number of observations n.

One of the most commonly used multitapers is the Discrete Prolate Spheroidal Sequences (DPSS) or tapers suggested by Thompson (1982), also known as Slepian sequences or tapers because the term DPSS was given in Slepian (1978). The Slepian tapers are sequences of functions, which are the Fourier transform of the DPSS, that were designed to maximize the spectral concentration defined as

(9.63)

for a frequency W, ∣W ∣  < π/2 is the bandwidth defining local and normally in the order of 1/n, where Consequently, the first of the sequences, {h₁(t), t = 1, …, n}, is chosen such that its corresponding spectral window or taper maximizing the concentration ratio in Eq. (9.63) over the interval (−W, W), which is equal to λ₁(n,W). This is done by maximizing the power

subject to the constraint that the total power is normalized such that

which leads to the following eigenvalue equation,

(9.64)

for t′ = 1, …, n. So, λ₁(n,W) is the largest eigenvalue and {h₁(t), t = 1, …, n} is the first taper. The second taper, {h₂(t), t = 1, …, n} is selected to maximize the corresponding concentration ratio but subject to being orthogonal to the first. Going further, the third taper maximizes the concentration ratio, but subject to being orthogonal to the first two tapers. In general, the jth taper corresponds to the jth largest eigenvalue, λ_j(n,W).

By maximizing concentration ratio, we are essentially minimizing the sidelobe energy outside a frequency band (−W,W). The eigenvalues should be close to unity for well‐behaved tapers. Let M be the matrix formed by those orthogonal tapers. Then M is an n × n positive–definite matrix with K dominant eigenvalues, λ₁(n,W) > λ₂(n,W) > ⋯ > λ_K(n,W) > ⋯ > λ_n(n,W), which are close to 1.

Another widely used set of tapers is called the Sine tapers (Riedel and Sidorenko, 1995), which are in the form of

(9.65)

for t = 1, …, n. They were used by Dai and Guo (2004) to obtain a preliminary estimate of the spectral matrix in the smoothing spline framework discussed in the next section.

9.3.3 Smoothing spline

The main disadvantage of the kernel smoothing method and the multitaper smoothing method is that they cannot guarantee that the final estimate is positive semidefinite while allowing flexible smoothing for each element of the spectral matrix. Thus, the same bandwidth is often applied to smoothing all the spectral components. However, in many applications, different components of the spectral matrix may need different smoothnesses, and require different smoothing parameters to get optimal estimates. To overcome this difficulty, Dai and Guo (2004) proposed a Cholesky decomposition based smoothing spline method for the spectrum estimation. The method models each Cholesky component separately by using different smoothing parameters. The method first obtains positive–definite and asymptotically unbiased initial spectral estimator through sine multitapers as shown in Eq. (9.65). Then, it further smooths the Cholesky components of the spectral matrix via the smoothing spline and penalized sum of squares, which allows different degrees of smoothness for different Cholesky elements.

Suppose the spectral matrix has Cholesky decomposition such that , where Γ is m × m lower triangular matrix. To obtain unique decomposition, the diagonal elements of Γ are constrained to be positive. The diagonal elements γ_j,j, j = 1, …, m, the real part of γ_j,k, ℜ(γ_j,k), and imaginary part of γ_j,k, ℑ(γ_j,k), j > k are smoothed by spline with different smoothing parameters. Suppose γ ∈ {γ_j,j, ℜ(γ_j,k), ℑ(γ_j,k), j > k, for j,k = 1, …, m}, we have

(9.66)

where a(ω_ℓ) = E{γ(ω_ℓ)}, and the e(ω_ℓ), ℓ = 1, …, n,are independent errors with zero means and the variances depending on the frequency point ω_ℓ. a(⋅) is periodic and is fitted by periodic smoothing spline (Wahba, 1990) of the form,

(9.67)

The estimator of power spectrum component is obtained by minimizing the following

(9.68)

where because of issues related asymptotic distributions, ω_ℓ = 0, 0.5,  and 1 are not used in the estimation, τ = n − 1 if n is odd, and τ = n − 2 if n is even; is the estimator of γ(ω_ℓ); and λ is a smoothing parameter. The final form of the estimator is given by

(9.69)

where [⋅] is the greatest integer function. The smoothing parameter λ can be chosen by the generalized cross‐validation or the generalized maximum likelihood. For more details, we refer readers to Wahba (1990) and Dai and Guo (2004). Qin and Wang (2008) suggested, through simulations, that the generalized maximum likelihood is stable and performs better than the generalized cross‐validation.

9.3.4 Bayesian method

Recall that the discrete Fourier transform , ℓ = 0, …, (n − 1)/2, are approximately independent complex multivariate normal random variables. The large‐sample distribution of leads to the Whittle likelihood (Whittle, 1953, 1954),

(9.70)

where ω_k = ℓ/n and L = [(n − 1)/2]. Based on the Whittle likelihood, Rosen and Stoffer (2007) proposed a Bayesian method to the estimate spectrum of the second‐order stationary multivariate time series. They model the spectrum f(ω) by the modified complex Cholesky factorization, such that,

(9.71)

where Γ_ℓ is a complex‐valued lower triangular matrix with one on its diagonal,

and D_ℓ is a diagonal matrix with positive real values, that is, . Let Θ = (θ₁, …, θ_L), and D = {D₁, …, D_L},the modified Cholesky representation facilitates development of Bayesian sampler by noticing that the Whittle likelihood can be rewritten as

(9.72)

where θ_ℓ is an m(m − 1)/2–dimensional vector and R_ℓ is a m × m(m − 1)/2 design matrix such that

and Y_{j, ℓ} is the jth entry of . Let and Each component of the Cholesky decomposition is fitted by the Demmler–Reinsch basis functions for linear smoothing splines of Eubank and Hsing (2008) as follows

(9.73)

for each frequency ω_ℓ. Let X be the design matrix of the basis functions and be the associated parameters. We then have

(9.74)

for j = 1, …, m, k = 1, …, j − 1. The priors on c_j, c_{j,k, (re)}, and c_{j,k, (im)} are chosen to be bivariate normal distributions and the priors on d_j, d_{j,k, (re)}, and d_{j,k, (im)} are chosen to be L–dimensional normal distributions , , and , respectively. The hyperparameters , and are smoothing parameters, which control the amount of smoothness. As the smoothing parameters tend to zero, the spline becomes a linear fit; as the smoothing parameters tend to infinity, the spline will be an interpolating spline. Gibbs sampling with the Metropolis–Hastings algorithm is used to draw parameters from posterior distribution.

9.3.5 Penalized Whittle likelihood

Penalized methods have been one of the most popular research topics for the past decade. In the area of spectrum estimation, Pawitan and O'Sullivan (1994) developed penalized likelihood method for the nonparametric estimation of the power spectrum of a univariate time series. Pawitan (1996) developed a penalized Whittle likelihood estimator for a bivariate time series. However, their approach cannot be easily generalized to higher dimension. More recently, Krafty and Collinge (2013) proposed a penalized Whittle likelihood method to estimate the power spectrum of a vector‐valued time series. The method allows for varying levels of smoothness among spectral components while accounting for the positive definiteness of spectral matrices and the Hermitian and periodic structures of power spectra as functions of frequency. In this section, we briefly discuss the method by Krafty and Collinge (2013).

Recall that are discrete Fourier transform of the time series, and define the negative log Whittle likelihood as

(9.75)

where ω_ℓ = ℓ/n and L = [(n − 1)/2]. Based on L, the method penalizes the roughness of the estimated spectrum. The spectral matrix f(ω) is modeled by the Cholesky decomposition such that f(ω) = [ΓΓ^*]⁻¹, where Γ is a m × m lower triangular matrix with real‐valued diagonal elements. Further, denote Γ_i,j,R(ω;f) and Γ_i,j,I(ω;f) as the real and imaginary parts of the (i,j) element of Γ. The method proposes a measure of roughness of a power spectrum through the integrated squared kth derivatives of the m(m + 1)/2 real and m(m − 1)/2 imaginary components of the Cholesky decomposition. Suppose the smoothing parameters, λ = {ρ_i,j, θ_i,j : i ≤ j = 1, …, m},of Γ_i,j,R and Γ_i,j,I that control the roughness penalty are ρ_i,j > 0 and θ_i,j > 0, the roughness measure for a spectrum is

(9.76)

The interpretation is that the penalty function J shrinks the estimates of power spectra toward real‐valued matrix functions that are constant across frequency. Consequently, we consider minimizing the penalized Whittle negative loglikelihood

The estimation procedure is based on an iterative algorithm adopted from Wood (2011) and the confidence intervals are obtained via bootstrap.

9.3.6 VARMA spectral estimation

As shown in Chapter 2, the underlying vector time series process is often described by a vector autoregressive moving average (VARMA) model. Specifically, the m‐dimensional VARMA process of order p and q, VARMA(p, q) is given by

(9.77)

or

(9.78)

where a_t is a sequence of m‐dimensional vector white noise process, VWN(0, Σ), and

(9.79)

We assume that all zeros of ∣Φ_p(B)∣ and ∣Θ_q(B)∣ lie outside of the unit circle, so that we can also represent it as

(9.80)

where and the sequence Ψ_j is square summable. The spectral density matrix of VARMA(p, q) model is given by

(9.81)

where and Ψ^*(e^−iω) is its conjugate transpose.

Given a vector time series of n observations, Z = (Z₁, Z₂, …, Z_n), we will first build its VARMA(p,q) model including the estimation of its parameter matrices. Let and be the corresponding estimates of the parameter matrices. We have

(9.82)

where and a_t is a sequence of m‐dimensional vector white noise, Then, the spectral density matrix estimation of the underlying process is given by

(9.83)

where

(9.84)

In practice, given a vector time series Z_t, t = 1, 2, …, n, one can often approximate the underlying process with a VAR(p) model,

(9.85)

So, the commonly used parametric estimation of spectral matrix is

(9.86)

where the order p can be determined through some kind of model selection criteria such as Akaike’s information criterion (AIC) or the Bayesian information criterion (BIC).

9.4.1 Sample spectrum

Let us first examine the sample spectrum of each series, which as shown in Wei (2006) and Priestley (1981) can be used to test hidden periodic components. They are displayed in Figure 9.2 and they are noisy.

So, we will compute a smoothed kernel sample spectrum matrix

(9.88)

where

(9.89)

where W(ω) is a spectral window, and M is the corresponding bandwidth. Let us simply consider Daniell's window, that is,

(9.90)

where

(9.91)

The smoothed five individual sample spectra with M = 5 are shown on the first row in Figure 9.3. We then increase the truncation point M to 7. The corresponding five smoothed sample spectra are shown on the second row in Figure 9.3, and they are much smoother.

It is clear that, except AUT, almost all other estimated spectra contain a large peak at two cycles per year, which corresponds to a half‐year periodic pattern. In addition, the estimated spectra of AUT and BUM have a peak at one cycle per year that is related to a yearly pattern. Finally, the spectra of GEM and COM also suggest a peak at four cycles per year, which is related to seasonal effects. In general, the power of AUT and GRO concentrates at higher frequencies; the power of BUM concentrates at lower frequencies; and the power of GEM and COM seems evenly spread out from medium to high frequencies.

From Eqs. (9.31) and (9.32), we have

(9.92)

and

(9.93)

Once the smoothed are obtained, we can compute the estimated co‐spectra, and the estimated quadrature spectra, as follows.

(9.94)

and

(9.95)

for i = 1, …, 5 and j = 1, …, 5. For an illustration, we use Daniell window with M = 7, and Figure 9.4 shows the estimated co‐spectra, and quadrature spectra, for j = 1, …, 4.

The estimated squared coherences can be obtained by

(9.96)

They are shown in Figure 9.5. An unexpected observation is that the squared coherence between COM and GEM is large and is constant across all frequencies. This indicates a strong relationship between sales changes of consumer materials and general merchandise.

9.4.2 Bayesian method

9.4.3 Penalized Whittle likelihood method

9.4.4 Example of VAR spectrum estimation

According to Eq. (9.86), the estimated spectral matrix is

(9.98)

where

Figure 9.8 shows the estimated spectrum from the VAR(4) model. Comparing with the kernel smoothed estimations given in Figure 9.3, we see that there are some similarities but also some differences. Similar to the nonparametric estimation, only the spectra of GEM and COM have peak at four cycles per year, which is related to seasonal effects. BUM concentrates at lower frequencies, GRO concentrates at higher frequencies, and the powers of GEM and COM are more evenly spread out in frequencies. From Figure 9.3, we see that the estimated spectra contain a large peak at two cycles per year except AUT. However, in terms of the VAR estimation shown in Figure 9.8, the estimated spectra contain a large peak at two cycles per year including AUT. Also, all estimated spectra have peak at one cycle per year, which is related to a yearly pattern, and unlike the kernel estimation, only AUT and BUM have peak at one cycle per year. Overall, the estimates of kernel window smoothing are much smoother than that of the VAR model. One possible reason is that the sample size of this particular example is small, which leads to unstable VAR parameter estimation.

Figure 9.9 shows the estimated co‐spectra, and quadrature spectra, for j = 1, …, 4. Comparing with the kernel smoothed estimations given in Figure 9.4, again we see that there are some similarities and differences. However, the VAR estimation of the squared coherences as shown in Figure 9.10 is quite similar to the kernel estimation from Figure 9.5. Both estimation methods show that the estimated squared coherence between COM and GEM is large and constant across all frequencies, indicating a strong relationship between sales changes of consumer materials and general merchandise.

9.5.1 Introduction

For a nonstationary time series, we normally use some transformation like variance stabilization and/or differencing to reduce it to stationary before performing its spectral matrix estimation. However, there are many kinds of nonstationary time series that cannot be reduced to stationary by these transformations. Let us first consider a univariate case. There are many univariate nonstationary processes Z_t, which cannot be represented by given in Eq. (9.1), because the function ϕ(ω) = e^iωt as a sine and cosine waves is stationary. Priestley (1965, 1966, and 1967) has pointed out that in this case, instead of using ϕ(ω) = e^iωt, we need to consider an oscillatory function, which is a generalized Fourier transform,

(9.99)

so that

(9.100)

where A(t,ω) is a time‐varying modulating or transfer function with absolute maximum at zero frequency. In other words, Z_t is an oscillatory process with an evolutionary spectrum, which has the same type of physical interpretation as the spectrum of a stationary process. The main difference is that while the spectrum of a stationary process describes the power distribution across frequencies over all time, the evolutionary spectrum describes power distribution over frequency at instantaneous time. However, within this framework, letting the length of series n → ∞ does not increase our knowledge about local behavior of spectrum since the pattern of the forthcoming series is different. As a result, the formulation does not allow the development of rigorous asymptotic theory of statistical inference. To overcome this problem, Dahlhaus (1996, 2000) introduced locally stationary time series that allows theoretical asymptotic analysis of the evolutionary spectrum for a univariate case, and further extended it to the multivariate case.

9.5.2 Spectrum representations of a nonstationary multivariate process

Let {Z_t : t = 1, …, n} be a m‐dimensional time series. The idea of locally stationary process is to rescale the transfer function to unit time scale, such that

(9.101)

More rigorously, we have following definition.

Based on Definition 9.1, the time‐varying power spectrum of the process is given by

(9.103)

Developed from locally stationary time series, methods for estimating the time‐varying spectrum of a multivariate time series can be roughly grouped into three categories. The first category consists of estimators in which second‐order frequency domain structures evolve continuously over time. This includes the slowly evolving multivariate locally stationary process, which can be analyzed parametrically by fitting time series models with time‐varying parameters (Dahlhaus, 2000), and nonparametrically via the bivariate smoothing of spectral components as functions of frequency and time (Guo and Dai, 2006). The second category of methods consist of estimators that are piecewise stationary. Approaches within this second category typically divide a time series into approximately stationary segments, then obtain the estimates of local spectrum within segments. These methods include both parametric approaches, such as fitting piecewise vector autoregressive models (Davis et al., 2006), and nonparametric approaches, such as using the multivariate smooth localized complex exponential (SLEX) library (Ombao von Sachs and Guo, 2005). The final category are methods that can automatically approximate both abrupt and slowly varying changes by averaging over piecewise stationary models, and they include the smoothing stochastic approximation Monte Carlo (SSAMC) based method of Zhang (2016) and the reversible jump Monte Carlo and Hamiltonian Monte Carlo (HMC) based method by Li and Krafty (2018).

9.5.2.1 Time‐varying autoregressive model

One of the methods used in Dahlhaus (2000) is the time‐varying VARMA(p,q) model. For a vector autoregressive model VAR(p), it is defined by

(9.104)

where ε_t are m‐dimensional independent random variable with mean zero and unit variance I. In addition, we assume some smoothness conditions on Σ(⋅) and Φ_j(⋅). In some neighborhood of a fixed time point u₀ = t₀/n, the process Z_t can be approximated by the stationary process Z_t(u₀) given by

(9.105)

Z_t has a unique time‐varying power spectrum, which is locally the same as the power spectrum of Z_t(u), that is,

(9.106)

where u = t/n, and

Similarly, the locally covariance matrix is

(9.107)

Based on this statement, the time‐varying spectrum can be obtained by estimating the time‐varying parameters of the VAR model. For more properties of the estimation based on time‐varying VARMA(p,q), VAR(p), and VMA(q) models, we refer readers to Dahlhaus (2000).

9.5.2.2 Smoothing spline ANOVA model

Based on the locally stationary process, the time‐varying spectrum can also be estimated nonparametrically via the smoothing spline Analysis of Variance (ANOVA) model by Guo and Dai (2006). However, their definition of locally stationary process is slightly different from Dahlhaus (2000). In Section 9.5.2, we mentioned that Dahlhaus (2000) assumes a series of transfer functions A⁰(t/n,ω) that converge to a large‐sample transfer function A(u,ω) in order to allow for the fitting of parametric models. Since Guo and Dai (2006) considered a nonparametric estimation, they used A(u,ω) directly.

Based on this definition, the smoothing ANOVA model takes a two‐stage estimation procedure. At the first stage, the locally stationary process is approximated by piecewise stationary time series with small blocks to obtain initial spectrum estimates and the Cholesky decomposition. The initial spectrum estimates are obtained by the multitaper method to reduce variance. At the second stage, each element of the Cholesky decomposition is treated as a bivariate smooth function of time and frequency and is modeled by the smoothing spline ANOVA model by Gu and Wahba (1993). The final estimated time‐varying spectrum is reconstructed from the smoothed elements of the Cholesky decomposition. Thus, the method provides a way to ensure the final estimate of the multivariate spectrum is positive–definite while allowing enough flexibility in the smoothness of its elements.

We shall briefly discuss the smoothing spline ANOVA step. Suppose the spectral matrix has the Cholesky decomposition such that where L(u,ω) is a m × m lower triangular matrix. The method smooths the diagonal elements γ_j,j(u,ω), j = 1, …, m, the real part of γ_j,k(u,ω), ℜ{γ_j,k(u,ω)}, and the imaginary part of γ_j,k(u,ω), ℑ{γ_j,k(u,ω)}, for j > k separately with their own smoothing parameters. Let γ(u,ω) ∈ {γ_j,j(u,ω), ℜ(γ_j,k)(u, ω), ℑ(γ_j,k)(u,ω), j > k, for j, k = 1, …, m}. We have

where a(u,ω) is the corresponding Cholesky decomposition element of the spectrum, such that a(u,ω) = E{γ(u,ω)}, the ε(u,ω) are independent errors with zero‐mean and the variance depending on the time‐frequency point (u,ω).

The smoothing spline ANOVA model is defined by the corresponding reproducing kernels (RKs) for the time and frequency domains. In the frequency domain, the reproducing kernel Hilbert space (RKHS) W₁ for ω can be decomposed as W₁ = 1 ⊕ H₁. The reproducing kernel for H₁ is R₁(ω₁, ω₂) =  − K₄(∣ω₁ − ω₂∣)/24, where K_k(⋅) is the kth order Bernoulli polynomial. In the time domain, the RKHS W₂ can be decomposed as W₂ = 1 ⊕ (u − 0.5) ⊕ H₂. The reproducing kernel for H₂ is R₂(u₁, u₂) = K₂(u₁)K₂(u₂)/4 − K₄(∣u₁ − u₂∣)/24. The full tensor product RKHS for (u,ω) is then

(9.108)

(9.109)

(9.110)

where the RK for H₃ = H₁ ⊗ (u − 0.5) is R₃{(u₁, ω₁), (u₂, ω₂)} = R₁(ω₁, ω₂)(u₁ − 0.5)(u₂ − 0.5), the RK for H₄ = H₁ ⊗ H₂ is R₄{(u₁, ω₁), (u₂, ω₂)} = R₁(ω₁, ω₂)R₂(u₁, u₂). Thus, a(u,ω) has the ANOVA‐like decomposition

(9.111)

where d₁ + d₂(u − 0.5) is the linear trend, a₁(ω) is the smooth main effect for frequency, a₂(u) is the smooth main effect across time, a₃(u,ω) is the smooth in frequency and linear in time, and a₄(u,ω) is the interaction that is smooth at both time and frequency. The least square estimation can be used to obtain the estimates For the details, see Guo and Dai (2006).

9.5.2.3 Piecewise vector autoregressive model

Davis, Lee, and Rodriguez‐Yam (2006) considered modeling nonstationary time series using piecewise VAR processes. The number and locations of the piecewise VAR segments, and the orders of the corresponding VAR process, are assumed to be unknown. Based on the minimum description length (MDL) principle, the method penalizes complexity of the model, and thus provides the criteria to define the best fitting model. We refer readers to Rissanen (1989), Hansen and Yu (2001) for more details about the MDL principle.

Let Z_t be a m‐dimensional time series with k segments, and assume that there are partitions or changepoints δ = (δ₀, …, δ_k)′ with δ₀ = 0 and δ_k = n. The time series within the jth segment is modeled by VAR(p_j) process, such that

(9.112)

where are m × m dimensional coefficient matrices of the VAR process. We define the entire class of piecewise VAR models as M and a model from this class as F ∈ M. The principle of MDL is to find the best fitting model from M as the one that produces the shortest code length. The code length of an object is the amount of memory space required to store the data Z_t. The MDL has two components, a fitted model and the portion that is unexplained by . The later component can be defined as the residuals . Let be the code length of fitted model and be the code length of the residuals of the jth segment. Then, the total code length of the data can be decomposed to

(9.113)

The MDL principle suggests that a best fitting model by minimizing L( Z_t).

Let n_j be the sample size in the jth segment and δ be the information set about the partition of segments. Since the piecewise VAR model can be characterized by following parameters: k, δ , p = (p₁, …, p_k), and , we can further decompose by

(9.114)

(9.115)

The equality is due to the fact that the complete knowledge of δ implies complete knowledge of n₁, …, n_k. In order to store an integer I whose value is not bounded, approximately log₂I bits are needed. Thus, we have L(k) = log₂k, L(p_j) = log₂p_j, and . On the other hand, if integer I is upper‐bounded say I_U, then log₂I_U bits are needed. Since n_j is bounded by n, we have L(n_j) = log₂n and L(n₁, …, n_k) = klog₂n (Hansen and Yu, 2001). It is known that a maximum likelihood estimate of a real parameter computed from n observations can be effectively encoded with (1/2)log₂n bits. Because the total number of parameters in is (p_j + 1)m², we have

(9.116)

Combining the results, we have

(9.117)

Rissanen (1989) showed that the code length of is approximated by the negative of the loglikelihood of the fitted model . Thus, we have

(9.118)

where L denotes the Whittle likelihood. Consequently, from Eqs. (9.114) and (9.115), the MDL objective function is defined as

(9.119)

The best fitting model is the one that minimizes the MDL. To overcome difficulties raised in optimization, the method proposes using a genetic algorithm.

9.5.2.4 Bayesian methods

There are two recently proposed Bayesian methods to estimate multivariate time‐varying spectrum, including Zhang (2016) and Li and Krafty (2018). In this section, we focus on the method of Li and Krafty (2018). The method is also based on locally stationary time series to estimate the time‐varying spectrum

(9.120)

The method assumes that for every u ∈ (0, 1), each component of f(u, ⋅) possesses a square‐integrable first derivative as a function of frequency; for every ω, each component of f(⋅, ω) is continuous as a function of scaled time at all but a possible finite number of points. The assumption on transfer function and time‐varying spectrum are slightly different from Definitions 9.1 and 9.2 in two ways. First, Definition 9.1 assumes a series of transfer functions A⁰(u,ω) that converge to a large‐sample transfer function A(u,ω) in order to allow for the fitting of parametric models. Since the method considers nonparametric estimation, in a manner similar to the smoothing ANOVA method, A(u,ω) is used. Second, Definitions 9.1 and 9.2 require the time‐varying spectrum to be continuous in both time and frequency. The method is more flexible and allows for components of spectrum to evolve not only continuously, but also abruptly in time.

The analysis of a locally stationary time series { Z_t : t = 1, …, n} begins by using the piecewise stationary approximation. Consider a partition of the time series into k segments defined by partition points δ = (δ₀, …, δ_k) with δ₀ = 0 and δ_k = n such that Z_t is approximately stationary within the segments {t : δ_q − 1 < t ≤ δ_q} for q = 1, …, k. Then

(9.121)

where A_q(ω) = A(u_q, ω)I(δ_q − 1 < t ≤ δ_q), I(⋅) is the indicator function, and u_q = (δ_q + δ_q − 1)/2n is the scaled midpoint of the qth segment. Within the qth segment, the time series is approximately second‐order stationary with local power spectrum f(u_q, ω) = A_q(ω)A_q(ω)^*.

Conditional on an approximately stationary partition δ, known approaches and properties for stationary time series can be applied. In particular, the large‐sample distribution of the discrete Fourier transform of a stationary time series that provides the Whittle likelihood allows for the formulation of a product of Whittle likelihoods. Let us define the local discrete Fourier transform at frequency ℓ within segment q as

(9.122)

where n_q = δ_q − δ_q − 1 is the number of observations in segment q, ω_q,ℓ = ℓ/n_q are the Fourier frequencies, and L_q = [(n_q − 1)/2].Given a partition of k segments, δ, the y_{q, ℓ} are approximately independent zero‐mean complex multivariate Gaussian random variables with covariance matrices, which equals spectral matrices f(u_q, ω_q,ℓ). This leads to a loglikelihood that can be approximated by a sum of log Whittle likelihoods

(9.123)

where Y is collection of Fourier transformations.

The spectral matrix is positive–definite and, to nonparametrically allow for flexible smoothing among the different components while preserving positive definiteness, the procedure modified the Cholesky components of local spectra via linear penalized splines. The modified Cholesky decomposition represents a time‐varying spectral matrix as

(9.124)

for a complex‐valued m × m lower triangular matrix Θ(u,ω) with ones on the diagonal and a positive diagonal matrix Ψ(u,ω). For a piecewise stationary approximation with partition δ into k segments, we define the local modified Cholesky decomposition as f⁻¹(u_q, ω) = Θ(u_q, ω)Ψ(u_q, ω)⁻¹Θ(u_q, ω)^*, for q = 1, …, k, and let θ_j,k,q(ω) and ψ_j,j,q(ω) be the (j,k) and (j,j) elements of Θ(u_q, ω) and Ψ(u_q, ω), respectively. Then, for each segment, there are m² components to estimate: ℜ{θ_j,k,q(ω)} for (j > k) = 1, …, m,ℑ{θ_j,k,q(ω)} for (j > k) = 1, …, m, and ψ_j,j,q(ω) for j = 1, …, m. The Cholskey components are modeled by periodic even and odd linear splines by considering

(9.125)

(9.126)

and

(9.127)

The Fourier frequencies for each segment form an equally spaced grid, so that Demmler–Reinsch bases for periodic even and odd smoothing splines for local periodograms are given by {cos(2πsω) : s = 0, 1, …, (L_q − 1)} and {sin(2πsω) : s = 1, …, L_q}, respectively. For the details, we refer readers to Schwarz and Krivobokova (2016).

The method relies on reversible jump Markov chain and HMC methods to sample from posterior distributions. The estimates of time‐varying spectrum components are obtained by averaging over the distribution of partitions so that both abrupt and slowly varying changes can be recovered.