In Section 6.6, we saw that a symmetric matrix is positive definite if and only if its eigenvalues are all positive. These types of matrices occur in a wide variety of applications. They frequently arise in the numerical solution of boundary value problems by finite difference methods or by finite element methods. Because of their importance in applied mathematics, we devote this section to studying their properties.

Recall that a symmetric $n\times n$ matrix A is positive definite if ${\mathbf{x}}^{T}A\mathbf{x}>0$ for all nonzero vectors x in ${ℝ}^n$. In Theorem 6.6.2, symmetric positive definite matrices were characterized by the condition that all their eigenvalues are positive. This characterization can be used to establish the following properties:

• Property I If A is a symmetric positive definite matrix, then A is nonsingular.

• Property II If A is a symmetric positive definite matrix, then $\text{det}\left(A\right)>0$.

If A were singular, $\lambda =0$ would be an eigenvalue of A. However, since all the eigenvalues of A are positive, A must be nonsingular. The second property also follows from Theorem 6.6.2, since

Given an $n\times n$ matrix A, let $A_r$ denote the matrix formed by deleting the last $n-r$ rows and columns of $A\mathrm{.}{A}_r$ is called the leading principal submatrix of A of order r. We can now state a third property of positive definite matrices:

• Property III If A is a symmetric positive definite matrix, then the leading principal submatrices $A_1,A_2,\mathrm{.}\text{ . . },A_n$ of A are all positive definite.

Proof

To show that $A_r$ is positive definite, $1\le r\le n$, let ${\text{x}}_r={(x_1,\mathrm{.}\text{ . . ,}{x}_r)}^T$ be any nonzero vector in ${ℝ}^r$ and set

Since

it follows that $A_r$ is positive definite.

An immediate consequence of properties I, II, and III is that if $A_r$ is a leading principal submatrix of a symmetric positive definite matrix A, then $A_r$ is nonsingular and $\text{det}\left(A_r\right)>0$ This has significance in relation to the Gaussian elimination process. In general, if A is an $n\times n$ matrix whose leading principal submatrices are all nonsingular, then A can be reduced to upper triangular form using only row operation III; that is, the diagonal elements will never be 0 in the elimination process, so the reduction can be completed without interchanging rows.

• Property IV If A is a symmetric positive definite matrix, then A can be reduced to upper triangular form using only row operation III, and the pivot elements will all be positive.

Let us illustrate property IV in the case of a $4\times 4$ symmetric positive definite matrix A. Note first that

so $a_{11}$ can be used as a pivot element and row 1 is the first pivot row. Let $a_{22}^{\left(1\right)}$ denote the entry in the (2, 2) position after the last three elements of column 1 have been eliminated (see Figure 6.7.1). At this step, the submatrix $A_2$ has been transformed into a matrix:

Figure 6.7.1.

Since the transformation was accomplished using only row operation III, the value of the determinant remains unchanged. Thus,

and hence

Since $a_{22}^{\left(1\right)}\ne 0$, it can be used as a pivot in the second step of the elimination process. After step 2, the matrix $A_3$ has been transformed into

Because only row operation III was used,

and hence

Thus, $a_{33}^{\left(2\right)}$ can be used as a pivot in the last step. After step 3, the remaining diagonal entry will be

In general, if an $n\times n$ matrix A can be reduced to an upper triangular form U without any interchanges of rows, then A can be factored into a product LU, where L is lower triangular with 1’s on the diagonal. The $(i,j)$ entry of L below the diagonal will be the multiple of the ith row that was subtracted from the jth row during the elimination process. We illustrate with a 3 × 3 example:

Given an LU factorization of a matrix A, it is possible to go one step further and factor U into a product $D{U}_1$, where D is diagonal and $U_1$ is upper triangular with 1’s on the diagonal:

It follows, then, that $A=LD{U}_1$. The matrices L and $U_1$ are referred to as unit triangular matrices since they are triangular and their diagonal entries are all equal to 1. The representation of a square matrix A as a product of the form LDU, where L is a unit lower triangular matrix, D is diagonal, and U is a unit upper triangular matrix, is referred to as an LDU factorization of A. In general, if A has an LDU factorization, then it is unique (see Exercise 8 at the end of this section).

If A is a symmetric positive definite matrix, then A can be factored into a product $LU=LD{U}_1$. The diagonal elements of D are the entries $u_{11},\dots ,u_{nn}$, which were the pivot elements in the elimination process. By property IV, these elements are all positive. Furthermore, since A is symmetric,

It follows from the uniqueness of the LDU factorization that $L^T=U_1$. Thus,

This important factorization is often used in numerical computations. There are efficient algorithms that make use of the $LD{L}^T$ factorization in solving symmetric positive definite linear systems.

• Property V If A is a symmetric positive definite matrix, then A can be factored into a product $LD{L}^T$, where L is lower triangular with 1’s along the diagonal and D is a diagonal matrix whose diagonal entries are all positive.

Since the diagonal elements $u_{11},\mathrm{.}\text{ . . },u_{nn}$ are positive, it is possible to go one step further with the factorization. Let

and set $L_1=L{D}^{1/2}$. Then

This factorization is known as the Cholesky decomposition of A.

• Property VI (Cholesky Decomposition) If A is a symmetric positive definite matrix, then A can be factored into a product $L{L}^T$, where L is lower triangular with positive diagonal elements.

The Cholesky decomposition of a symmetric positive definite matrix A can also be represented in terms of an upper triangular matrix. Indeed, if A has Cholesky decomposition $L{L}^T$ where L is lower triangular with positive diagonal entries, then the matrix $R=L^T$ is upper triangular with positive diagonal entries and

The Cholesky factorization of the symmetric positive definite matrix A in Example 3 could also have been written in terms of the upper triangular matrix $R=L_1^T$.

More generally, it is not difficult to show that any product of the $B^{T}B$ will be positive definite, provided that B is nonsingular. Putting all these results together, we have the following theorem.

Proof

We have already shown that (a) implies (b), (b) implies (c), and (c) implies (d). To see that (d) implies (e), assume that $A=L{L}^T$. If we set $B=L^T$, then B is nonsingular and

Finally, to show that $\left(\text{e}\right)\Rightarrow \left(\text{a}\right)$, assume that $A=B^{T}B$, where B is nonsingular. Let x be any nonzero vector in ${ℝ}^n$ and set $\mathbf{y}=B\mathbf{x}$. Since B is nonsingular, $\mathbf{y}\ne \mathbf{0}$ and it follows that

∎

Thus, A is positive definite.

Analogous results to Theorem 6.7.1 are not valid for positive semidefiniteness. For example, consider the matrix

The leading principal submatrices all have nonnegative determinants:

However, A is not positive semidefinite, since it has a negative eigenvalue $\lambda =-1$. Indeed, $\mathbf{x}={(1,1,1)}^T$ is an eigenvector belonging to $\lambda =-1$ and