Kurtosis of the Marginal Distribution

In view of ( 12.3), the existence of a fourth‐order moment for ε _t requires the conditions

12.4

We then have

and the kurtosis coefficient of the marginal distribution of ε _t is given by

where κ _η is the kurtosis coefficient of η _t and κ ⁽ⁱ⁾ is the kurtosis coefficient of . Because the law of is non‐degenerate, κ ⁽ⁱ⁾ > 1. Thus, the kurtosis of the marginal distribution is strictly larger than that of the conditional distribution of ε _t . When , we have In particular if we also have , then the distribution of ε _t is leptokurtic ( κ _ε  > 3).

Tails of the Marginal Distribution

When stable laws are estimated on series of financial returns, one generally obtains heavy‐tailed marginal distributions. ⁴ It is, therefore, useful to study in more detail the tail properties of the marginal distribution of SV processes. We relax in this section the assumption of finite variance for η _t .

The distribution of a non‐negative random variable X is said to be regularly varying with index α , which can be denoted X ∈ RV(α) if

12.5

with α ∈ (0, 2) and L(x) a slowly varying function at infinity, which is defined by the asymptotic relation L(tx)/L(x) → 1 as x → ∞, for all t > 0. ⁵ If X satisfies (12.5), then E(X ^r ) = ∞ for r > α and E(X ^r ) < ∞ for r < α (for r = α , the expectation may be finite or infinite, depending on the slowly varying function L ). To handle products, the following result by Breiman (1965, Eq. (3.1)) can be useful: for any non‐negative independent random variables X and Y , where X ∈ RV(α) and E(Y ^α + δ ) < ∞ for some δ > 0, we have

It follows that if, in Model ( 12.1) without the assumption that , ∣η _t  ∣  ∈ RV(α), and if the noise (υ _t ) is such that , for some τ > 0, then ε _t inherits the heavy‐tailed property of the noise (η _t ). This is in particular the case if (υ _t ) is Gaussian, since h _t then admits moments at any order.

ARMA Representation for

Assuming P(η _t  = 0) = 0, and by taking the logarithm of , the first equation of ( 12.1) writes as follows:

12.6

This formulation allows us to derive the autocovariance function of the process . Let Assume that Z _t admits fourth‐order moments, and let μ _Z  = E(Z _t ), , and ⁶ The next result (see Exercise 12.3 for a proof) shows that (X _t ) admits an ARMA representation.

Note that, in Eq. (12.7), the noise (u _t ) is not strong. This sequence is not even a martingale difference sequence (unlike the noise of the ARMA representation for the square of a GARCH process). This can be seen by computing, for instance,

Even when ( ) is symmetrically distributed, this quantity is generally non‐zero. This shows that u _t is correlated with some (non‐linear) functions of its past: thus E(u _t  ∣ u _t − 1, u _t − 2, …) ≠ 0 (a. s.).

Autocovariance of the Process

When the process is second‐order stationary, that is under Eq. (12.4), deriving its autocovariance function has interest for comparison with GARCH models and for statistical applications.

Using Eq. ( 12.3) we get, for all k ≥ 0

For k > 0, since and are independent and have unit expectation,

where, for i ≥ k , α _{k, i}  = E[exp{σβ ⁱ (1 + β ^k ) }]. Thus, we have, for all k > 0

Now suppose that . It follows that , and thus the second‐order stationarity condition reduces to and ∣β ∣  < 1 in this case. Therefore, for any k > 0

It can be noted that the autocovariance function of tends to zero when k goes to infinity, but that the decreasing is not compatible with a linear recurrence equation on the autocovariances. In other words, the process – though second‐order stationary and admitting a Wold representation – does not have an ARMA representation. However, we have an equivalent of the form as k → ∞, which shows that the asymptotic (exponential) rate of decrease of the autocovariances is the same as for an ARMA process. Recall that, by contrast, squared fourth‐order stationary GARCH processes satisfy an ARMA model.

Contemporaneous Correlation

A contemporaneous correlation coefficient ρ between the disturbances of model ( 12.1) is introduced by assuming that (η _t , υ _t ) forms an iid centred normal sequence with Var(η _t ) = Var(υ _t ) = 1 and Cov(η _t , υ _t ) = ρ . The model can thus be equivalently written as

12.9

where ω _t  = ω + σρη _t , , (η _t ) and are two independent sequences of iid normal variables with zero mean and variance equal to 1. We still assume that ∣β ∣  < 1, which ensures stationarity of the solution (ε _t ) defined in ( 12.3). If the correlation ρ is negative, a negative shock on the price – corresponding to a negative η _t – is likely to be associated with an increase of the volatility at the current date and – if β > 0 – at the subsequent dates. Conversely, a positive shock on the price – η _t  > 0 – is likely to be compensated by a decrease of the volatility. The contemporaneous correlation between the iid processes of the SV model thus induces asymmetry effects, similar to the leverage effect described in Chapter 1. This effect, characterised by a negative correlation between the current price and the future volatility, has been often detected in stock prices. However, the correlation coefficient ρ also entails a non‐zero mean

and non‐zero autocorrelations for the process (ε _t ) which, in financial applications, is often inappropriate (see Exercise 12.4). Therefore, it appears that the correlation coefficient ρ has several roles in this model, which makes it difficult to interpret.

Leverage Effect

A non‐contemporaneous correlation between the disturbances of model ( 12.1) allows to pick up the leverage effect, without removing the zero‐mean property. Therefore, we assume that (η _t − 1, υ _t ) forms an iid centred normal sequence with Var(η _t ) = Var(υ _t ) = 1 and Cov(η _t − 1, υ _t ) = ρ . The model is given by

12.10

with the same notations as in the last section. By ( 12.3), we have E(ε _t ) = 0 and Cov(ε _t , ε _t − k ) = 0 for k > 0. Thus, the white noise property of the stationary solution is preserved with this correlation structure of the two disturbances sequences.

Intuitively, when ρ < 0, a large negative shock on the noise η at time t − 1 – that is, a sudden price fall – is likely to produce (through the correlation between and η _t − 1 ) a large volatility at the next period (and at the subsequent ones if β > 0). The following result shows that the model is indeed able to capture the leverage effect and provides a quantification of this effect.

To summarise, the SV model where η _t − 1 and υ _t are negatively correlated is able to capture the leverage effect, while preserving the white noise property of (ε _t ). As the horizon increases, however, this effect is maintained only if β > 0 and it vanishes exponentially.

Comparison with GARCH Models

As in GARCH or SV models, the observed process (ε _t ) is centred and its sample paths fluctuate around zero. The magnitude of the fluctuations will be different according to the state of the Markov chain Δ _t . Regime 1 is characterised by small fluctuations, while regime d is associated with the most turbulent periods. The length of stay in a given regime, as well as the number of transitions from one regime to another only depend on the transition probabilities of the Markov chain. An example of sample path (100 observations) is displayed in Figure 12.1. The path is simulated from a three‐regime model with and

This very simple model presents similarities with GARCH‐type models. By construction, the clustering property ⁸ is satisfied.

The assumptions made on the Markov chain and on ( η _t ) entail that any solution of model (12.15) is obviously strictly stationary. Moments of ε _t exist at any order, provided that the same moments exist for η _t . By the independence between η _t and Δ _t , we have for any real positive number r ,

In particular, ε _t is centred. Recall that for GARCH processes, moments cannot exist at any order. Although the marginal distribution of ε _t does not possess heavy‐tails, the leptokurticity property is satisfied in the HMM: the law of ε _t has fatter tails than that of η _t . If a Gaussian distribution is assumed for η _t , the Kurtosis coefficient of ε _t is strictly greater than 3 (the exact value depends on both the transition probabilities of the chain and the values of the variances ω(i) in the different regimes).

An important difference with GARCH model pops up when we compute the conditional Kurtosis coefficient. Recall that this coefficient is constant for GARCH‐type models. For model ( 12.15) we have

and no simplification occurs here, because is not a function of past values of ε _t . Thus, the shape of the conditional distribution – not only the variance – may also fluctuate over time.

Less straightforward similarities with standard GARCH models appear by studying the autocorrelation functions of (ε _t ) and .

Autocorrelations, Moments

We have

hence (ε _t ) is a white noise with variance

We now study the autocorrelation of the squared process. For ease of presentation, let us first consider the case where d = 2. The eigenvalues of the transition matrix are 1 and λ ≔ p(1, 1) + p(2, 2) − 1, since the Markov chain is, by assumption A1, irreducible and aperiodic. Note that −1 < λ < 1. By diagonalising ℙ, it can easily be seen that the entries of ℙ ^k have the form p ^(k)(i, j) = a ₁(i, j) + a ₂(i, j)λ ^k , for k ≥ 0. By letting k go to infinity, we get a ₁(i, j) = π(j) and, using the value k = 0: a ₁(i, j) + a ₂(i, j) =  _{i = j}. It follows that for j = 1, 2 and i ≠ j

hence, for i, j = 1, 2

12.17

We have, for k > 0,

12.18

and thus, by using Eq. (12.17),

12.19

It is worth noting that the autocorrelations of the squares decrease at exponential rate like for a second‐order stationary GARCH process. An important difference, however, is that the rate of this convergence is not related to the existence of moments. Note also that the larger ∣λ ∣  =  ∣ 1 − p(1, 1) − p(2, 2)∣, the slower the decrease of the autocorrelations (in module). In other words, the autocorrelations decrease slowly when the transition probabilities from a regime to the other are either both very small or both close to 1. Obviously, for any k > 0 when ω(1) = ω(2) because ε _t is an iid white noise in this case. A similar computation shows that

12.20

We deduce from Eqs. (12.19) and (12.20) that admits an ARMA(1,1) representation, with autoregressive coefficient λ _.

In the general case, matrix ℙ may not be diagonalisable but has an eigenvalue equal to 1, with corresponding eigenspace of dimension 1. Denote by λ ₁,…, λ _m the other eigenvalues and n ₁,…, n _m the dimensions of the corresponding eigenspaces ( n ₁ + ⋯ + n _m  = d − 1). The Jordan form of matrix ℙ is ℙ = SJS ⁻¹ , where S is non‐singular and

with J _l (λ) = λ핀 _l  + N _l (1), denoting by N _l (i) the square matrix of dimension l whose entries are equal to zero, except for the i th uperdiagonal whose elements are equal to 1. Using for k ^′ ≤ l − 1 and for k ^′ > l − 1, we get

where P ^(l) is a polynomial of degree l − 1. It follows that

We deduce that

where the are polynomials of degree n _l  − 1. The first term in the right‐hand side is identified by the convergence p ^(k)(i, j) → π(j) as k → ∞. Note that ∣λ _l  ∣  < 1 for l = 1,…, m . Finally, we have, using (12.18),

12.21

where the 's are polynomials of degree n _l  − 1.

The computation of is more tedious, but it can be shown that this expectation depends of k through the 's (similarly to (12.21)). Thus, the Markov chain (Δ _t ) introduces a source of persistence of shocks on the volatility.

Estimation

The vector of parameters of model ( 12.15) is

The likelihood can be written by conditioning on all possible paths (e ₁,…, e _n ) of the Markov chain, where the e _i belong to ℰ = {1,…, d}. For ease of presentation, we assume a standard Gaussian distribution for η _t but the estimation procedure can be adapted to other distributions. The probability of such paths is given by

For each path, we get a conditional likelihood of the form

where φ _i (·) denotes the density of the law .

Finally, the likelihood writes

Unfortunately, this formula cannot be used in practice because the number of summands is d ⁿ , which is huge even for small samples and two regimes. For instance, 2³⁰⁰ is generally considered to be an upper bound for the number of atoms in the universe. ⁹ Several solutions to this numerical problem – computing the likelihood – exist.

Computing the Likelihood

Matrix Representation

Let

where g _k (· ∣ Δ _k  = i) is the law of (ε ₁,…, ε _k ) given {Δ _k  = i}. One can easily check that

12.22

12.23

and

12.24

In matrix form we get

where

Hence, letting , we have

12.25

which is computationally usable (with an order of d ² n multiplications).

Forward–Backward Algorithm

Let B _k (i) = B _k (ε _k + 1,…, ε _n  ∣ Δ _k  = i) the law of (ε _k + 1,…, ε _n ) given {Δ _k  = i}. By the Markov property, we have

The Forward formulas, allowing us to compute F _k (i) for k = 1, 2, … , are given by (12.22) and (12.23). The Backward formulas, allowing us to compute B _k (i) for k = n − 1, n − 2, …  are

12.26

12.27

We thus obtain

12.28

for any k ∈ {1,…, n}. For k = n , we retrieve Eq. (12.24).

Hamilton Filter

The Forward–Backward algorithm was developed in the statistical literature (Baum 1972). Models involving a latent Markov chain were put forward in the econometric literature by Hamilton (1989). Let

φ(ε _t ) = (φ ₁(ε _t ),…, φ _d (ε _t ))^′ , and denote by ⊙ the element‐by‐element Hadamard product of matrices.

With obvious notations, we get

where

Starting from an initial value π _1 ∣ 0 = π (the stationary law) or π _1 ∣ 0 = π ₀ (a given initial law), we can compute

12.29

for t = 1,…, n , and we deduce the conditional log‐likelihood

12.30

where

12.31

The Hamilton algorithm (12.29)–(12.31) seems numerically more efficient than the Forward–Backward algorithm in ( 12.22)–( 12.23) and (12.26)–(12.28) which, under this form, produces underflows. Note, however, that conditional versions of the Forward–Backward algorithm have been developed to avoid this problem (Devijver 1985). Note also that the matrix formulation (12.25) is very convenient for establishing asymptotic properties of the ML estimator (see Francq and Roussignol 1997).

Maximising the Likelihood

Maximisation of the (log‐)likelihood can be achieved by a classical numerical procedure, or using the EM (Expectation–Maximisation) algorithm, which can be described as follows.

For ease of exposition, we consider that the initial distribution π ₀ (the law of Δ₁ ) is not necessarily the stationary distribution π . In the EM algorithm, π ₀ is an additional parameter to be estimated. If – in addition to (ε ₁,…, ε _n ) – one could also observe (Δ₁,…, Δ _n ), estimating θ and π ₀ by ML would be an easy task. Indeed,

where

12.32

12.33

12.34

From (12.32), we need to maximise the terms with respect to ω(i), for i = 1,…, d . This yields the estimators

12.35

Maximisation of (12.33) with respect to π ₀(1),…, π ₀(d), under the constraint , yields

12.36

From (12.34), for i = 1,…, d , one has to maximise with respect to p(i, 1),…, p(i, d), under the constraint , the sum

We obtain ¹⁰

12.37

In practice, formulas (12.35), (12.36), and (12.37) cannot be used, because (Δ _t ) is unobserved. Generally speaking, the EM algorithm alternates between steps E – for expectation – where the expectation of the likelihood is evaluated using the current value of the parameter, and steps M – for maximisation – where the objective function computed in step E is maximised. In the present setting, the EM algorithm only requires steps M, together with the computation of the predicted and filtered probabilities – π _{t ∣ t − 1} and π _t ∣ t , respectively – of the Hamilton filter ( 12.29), and also an additional step for the computation of the smoothed probabilities.

Step E

Suppose that an estimator de (θ, π ₀) is available. It seems sensible to approximate the unknown log‐likelihood by its expectation given the observations (ε ₁,…, ε _n ), evaluated under the law parameterised by . We get the criterion

where

12.38

12.39

12.40

Step M

We aim at maximising, with respect to (θ, π ₀), the estimated log‐likelihood . Maximisation of (12.38) yields the solution

12.41

The estimated variance of regime i is thus a weighted mean of the , where the weights are the conditional probabilities that the chain lie in state i at time t . Similarly, (12.39) yields

12.42

and (12.40) yields

12.43

Formulas (12.41), (12.42), and (12.43) require computation of the smoothed probabilities

and

Computation of Smoothed Probabilities

The Markov property entails that, given Δ _t , the observations ε _t , ε _t + 1, … do not convey information on Δ _t − 1 . We hence have

and

It remains to compute the smoothed probabilities π _t ∣ n , given by

for t = n, n − 1,…, 2. Starting from an initial value , formulas ( 12.41), ( 12.42), and ( 12.43) allow us to obtain a sequence of estimators which increase the likelihood (see Exercise 12.13).

Summary of the Algorithm

Starting from initial values for the parameters

the algorithm consists in

repeating the following steps until consistency:

1. Set π _1 ∣ 0 = π ₀ and
   
2. Compute the smoothed probabilities π _t ∣ n (i) = P(Δ _t  = i ∣ ε ₁,…, ε _n ) using
   

   and π _{t − 1, t ∣ n} (i, j) = P(Δ _t − 1 = i, Δ _t  = j ∣ ε ₁,…, ε _n ) from

   
3. Replace the previous values of the parameters by π ₀ = π _1 ∣ n ,
   

In practice, the sequence converges rather rapidly to the ML estimator (see the illustration below) if an appropriate starting value θ ⁽⁰⁾ > 0 is chosen (see Exercise 12.9).

Model ( 12.15) is certainly too simple to take into account, in their complexity, the dynamic properties of real‐time series (for instance of financial returns): whenever the series remain in the same regime, the observations have to be independent, and this assumption is quite unrealistic. A natural extension of model ( 12.15) – and also of the standard GARCH model – is obtained by assuming that, in each regime, the dynamics is governed by a GARCH model.

Illustration

For the sake of illustration, consider the CAC 40 and SP 500 stock market indices. Observations cover the period from March 1, 1990, to December 29, 2006. On the daily returns (in %), we fitted the HMM ( 12.15) and ( 12.16) with d = 4 regimes, using the R code given in Exercise 12.8. Taking initial values with non‐zero transition probabilities (see Exercise 12.9), after roughly 60 iterations of the EM algorithm, we obtain the following estimated model for the SP 500 series

and for the CAC 40

The estimated probabilities of the regimes are

and the average durations of the four regimes – given by 1/{1 − p(i, i)} – are approximately

Thus, the CAC stays in average 27 days in the more volatile regime, namely Regime 4. Figure 12.2 confirms that for the two series, the more volatile regime is also the less‐persistent regime, with however a long period of high volatility of 81 days (from June 27, 2002 to October 21, 2002) for the SP, and of 113 days (from June 4, 2002 to November 8, 2002) for the CAC. It is interesting to note that for the model of the SP 500, the possible transitions are always from one regime to an adjacent regime. Being, for instance in Regime 2, the chain can stay in Regime 2 or move to Regime 1 or 3, but the probability p(2, 4) of moving directly to Regime 4 is approximately zero. On the contrary, the CAC displays sudden moves from Regime 2 to Regime 4.

Conclusion

Estimation of GARCH‐type models over long time series, as those encountered in finance (i.e. typically several thousands of observations), generally produces strong volatility persistence. This effect may be spurious and may be explained by the need to obtain heavy‐tailed marginal distributions.

MS‐GARCH models enable to disentangle important properties of financial time series: persistence of shocks, decreasing autocorrelations of the squared returns, heavy tailed marginal distributions, and time‐varying conditional densities. These models are appropriate for series over a long time period, with successions of phases corresponding to the different regimes. Of course, such models – despite their flexibility – remain merely approximations of the real data generation mechanisms. ¹²

12.1 (Second‐order stationarity of the SV model)

Prove Theorem 12.2.