1.3. Multivariate Normal Distribution

A probability distribution that plays a pivotal role in multivariate analysis is multivariate normal distribution. We say that y has a multivariate normal distribution (with a mean vector μ and the variance-covariance matrix Σ) if its density is given by

In notation, we state this fact as y ~ N_p(μ, Σ). Observe that the above density is a straightforward extension of the univariate normal density to which it will reduce when p = 1.

Important properties of the multivariate normal distribution include some of the following:

• Let A_{r × p} be a fixed matrix, then Ay ~ N_r (Aμ, AΣA′) (r ≤ p). It may be added that Ay will admit the density if AΣA′ is nonsingular, which will happen if and only if all rows of A are linearly independent. Further, in principle, r can also be greater than p. However, in that case, the matrix AΣA′ will not be nonsingular. Consequently, the vector Ay will not admit a density function.

• Let G be such that Σ⁻¹ = GG′, then G′y ~ N_p(G′μ, I) and G′(y - μ) ~ N_p(0, I).

• Any fixed linear combination of y₁, ..., y_p, say c′y, c_p×1 ≠ o is also normally distributed. Specifically, c′y ~ N₁ (c′μ, c′Σc).

• The subvectors y1 and y2 are also normally distributed, specifically, y1 ~ N_p1 (μ₁, Σ₁₁) and y2 ~ N_p-p1(μ₂,Σ₂₂).

• Individual components y1, ..., y_p are all normally distributed. That is, y_i ~ N₁ (μ_i, σ_ii), i = 1, ..., p.

• The conditional distribution of y1 given y2, written as y1|y2, is also normal. Specifically,

  
  

  Let μ₁ + Σ₁₂Σ₂₂⁻¹(y2 - μ₂) = μ₁ - Σ₁₂Σ₂₂⁻¹μ₂ + Σ₁₂Σ₂₂⁻¹y2 = B₀ + B₁y2, and Σ_11·2 = Σ₁₁ - Σ₁₂ Σ₂₂⁻¹ Σ₂₁. The conditional expectation of y1 for given values of y2 or the regression function of y1 on y2 is B₀+B₁y2, which is linear in y2. This is a key fact for multivariate multiple linear regression modeling. The matrix Σ_11·2 is usually represented by the variance-covariance matrix of error components in these models. An analogous result (and the interpretation) can be stated for the conditional distribution of y2 given y1.

• Let δ be a fixed p × 1 vector, then

  
  

  The random components y1, ..., y_p are all independent if and only if Σ is a diagonal matrix; that is, when all the covariances (or correlations) are zero.

• Let u1 and u2 be respectively distributed as N_p (μ_u1, Σ_u1) and N_p(μ_u2, Σ_u2), then

  u1 ± u2 ~ N_p(μ_u1 ± μ_u2, Σ_u1 + Σ_u2 ± (cov(u1, u2) + cov(u2, u1))).

  Note that if u1 and u2 were independent, the last two covariance terms would drop out.

There is a vast amount of literature available on multivariate normal distribution, its properties, and the evaluations of multivariate normal probabilities. See Kshirsagar (1972), Rao (1973), and Tong (1990) among many others for further details.