1 INTRODUCTION

In this final chapter of Part One we shall discuss some more specialized topics, which will be applied later in this book. These include some further results on adjoint matrices (Sections 3.2 and 3.3), Hadamard products (Section 3.6), the commutation and the duplication matrix (Sections 3.7–3.10), and some results on the bordered Gramian matrix with applications to the solution of certain matrix equations (Sections 3.13 and 3.14).

2 THE ADJOINT MATRIX

We recall from Section 1.9 that the cofactor c_ij of the element a_ij of any square matrix A is (−1)^i+j times the determinant of the submatrix obtained from A by deleting row i and column j. The matrix C = (c_ij) is called the cofactor matrix of A. The transpose of C is called the adjoint matrix of A and we use the notation

(1)

We also recall the following two properties:

(2)

(3)

Let us now prove some further properties of the adjoint matrix.

Before giving the proof of Theorem 3.1, we formulate the following two consequences of this theorem.

A direct proof of Theorem 3.3 is given in the Miscellaneous Exercises 5 and 6 at the end of Chapter 8.

3 PROOF OF THEOREM 3.1

If r(A) = n, the result follows immediately from (2). To prove that A^# = 0 if r(A) ≤ n − 2, we express the cofactor c_ij as

where E_j is the n × (n − 1) matrix obtained from I_n by deleting column j. Now, is an (n − 1) × (n − 1) matrix whose rank satisfies

It follows that is singular and hence that c_ij = 0. Since this holds for arbitrary i and j, we have C = 0 and thus A^# = 0.

The difficult case is where r(A) = n − 1. Let λ₁, λ₂, …, λ_n be the eigenvalues of A, and assume

while the remaining n − k eigenvalues are nonzero. By Jordan's decomposition theorem (Theorem 1.14), there exists a nonsingular matrix T such that

(8)

where J₁ is the k × k matrix

J₂ is the (n − k) × (n − k) matrix

and δ_j (k + 1 ≤ j ≤ n − 1) can take the values 0 or 1 only.

It is easy to see that every cofactor of J vanishes, with the exception of the cofactor of the element in the (k, 1) position. Hence,

(9)

where e₁ and e_k are the first and kth elementary vectors of order n × 1, and μ(A) denotes the product of the n − k nonzero eigenvalues λ_k+1, …, λ_n of A. (An elementary vector e_i is a vector with one in the ith position and zeros elsewhere.) Using (3), (8), and (9), we obtain

(10)

Since the first column and the kth row of J₁ are zero, we have Je₁ = 0 and . Hence,

Further, since r(A) = n − 1, the vectors x and y satisfying Ax = A′y = 0 are unique up to a factor of proportionality, that is,

(11)

for some real α and β. Now,

and

It follows that

(12)

Hence, from (11) and (12),

(13)

Inserting (13) in (10) concludes the proof.

4 BORDERED DETERMINANTS

The adjoint matrix also appears in the evaluation of the determinant of a bordered matrix, as the following theorem demonstrates.

As one of many special cases of Theorem 3.4, we present Theorem 3.5.

5 THE MATRIX EQUATION AX = 0

In this section, we will be concerned in finding the general solutions of the matrix equation AX = 0, where A is an n × n matrix with rank n − 1.

6 THE HADAMARD PRODUCT

If A = (a_ij) and B = (b_ij) are matrices of the same order, say m × n, then we define the Hadamard product of A and B as

Thus, the Hadamard product A ⊙ B is also an m × n matrix and its ijth element is a_ij b_ij.

The following properties are immediate consequences of the definition:

(14)

(15)

(16)

so that the brackets in (16) can be deleted without ambiguity. Further,

(17)

(18)

(19)

where J is a matrix consisting of ones only.

The following two theorems are of importance.

7 THE COMMUTATION MATRIX K_mn

Let A be an m × n matrix. The vectors vec A and vec A′ contain the same mn components, but in a different order. Hence, there exists a unique mn × mn permutation matrix which transforms vec A into vec A′. This matrix is called the commutation matrix and is denoted by K_mn or K_m,n. (If m = n, we often write K_n instead of K_nn.) Thus,

(20)

Since K_mn is a permutation matrix, it is orthogonal, i.e. , see Equation ( 13 ) in Chapter 1. Also, premultiplying (20) by K_nm gives

so that K_nmK_mn = I_mn. Hence,

(21)

Further, using Equation ( 11 ) in Chapter 2,

The key property of the commutation matrix (and the one from which it derives its name) enables us to interchange (commute) the two matrices of a Kronecker product.

An important application of the commutation matrix is that it allows us to transform the vec of a Kronecker product into the Kronecker product of the vecs, a crucial property in the differentiation of Kronecker products.

Closely related to the matrix K_n is the matrix , denoted by N_n. Some properties of N_n are given in Theorem 3.11.

8 THE DUPLICATION MATRIX D_n

Let A be a square n × n matrix. Then vech(A) will denote the vector that is obtained from vec A by eliminating all supradiagonal elements of A (i.e. all elements above the diagonal). For example, when n = 3:

and

In this way, for symmetric A, vech(A) contains only the generically distinct elements of A. Since the elements of vec A are those of vech(A) with some repetitions, there exists a unique matrix which transforms, for symmetric A, vech(A) into vec A. This matrix is called the duplication matrix and is denoted by D_n. Thus,

(22)

Let A = A′ and D_n vech(A) = 0. Then vec A = 0, and so vech(A) = 0. Since the symmetry of A does not restrict vech(A), it follows that the columns of D_n are linearly independent. Hence, D_n has full column rank , is nonsingular, and , the Moore‐Penrose inverse of D_n, equals

(23)

Since D_n has full column rank, vech(A) can be uniquely solved from (22) and we have

(24)

Some further properties of D_n are easily derived from its definition ( 22 ).

Much of the interest in the duplication matrix is due to the importance of the matrices and , some of whose properties follow below.

Finally, we state, without proof, two further properties of the duplication matrix which we shall need later.

9 RELATIONSHIP BETWEEN D_n+1 AND D_n, I

Let A₁ be a symmetric (n + 1) × (n + 1) matrix. Our purpose is to express and as partitioned matrices. In particular, we wish to know whether is a submatrix of and whether is a submatrix of when A is the appropriate submatrix of A₁. The next theorem answers a slightly more general question in the affirmative.

10 RELATIONSHIP BETWEEN D_n+1 AND D_n, II

Closely related to Theorem 3.15 is the following result.

As a consequence of Theorem 3.16, we obtain Theorem 3.17.

11 CONDITIONS FOR A QUADRATIC FORM TO BE POSITIVE (NEGATIVE) SUBJECT TO LINEAR CONSTRAINTS

Many optimization problems take the form

and, as we shall see later (Theorem 7.12) when we try to establish second‐order conditions for Lagrange minimization (maximization), the following theorem is then of importance.

12 NECESSARY AND SUFFICIENT CONDITIONS FOR r(A : B) = r(A) + r(B)

Let us now prove Theorem 3.19.

13 THE BORDERED GRAMIAN MATRIX

Let A be a positive semidefinite n × n matrix and B an n × k matrix. The symmetric (n + k) × (n + k) matrix

called a bordered Gramian matrix, is of great interest in optimization theory. We first prove Theorem 3.20.

Next we obtain the Moore‐Penrose inverse of Z.

In the special case where ℳ(B) ⊂ ℳ(A), the results can be simplified. This case is worth stating as a separate theorem.

14 THE EQUATIONS X₁A + X₂B′ = G₁, X₁B = G₂

The two matrix equations in X₁ and X₂,

where A is positive semidefinite, can be written equivalently as

The properties of the matrix Z studied in the previous section enable us to solve these equations.

An important special case of Theorem 3.23 arises when we take G₁ = 0.

BIBLIOGRAPHICAL NOTES

2. A good discussion on adjoint matrices can be found in Aitken (1939, Chapter 5). Theorem 3.1(b) appears to be new.

6. For a review of the properties of the Hadamard product, see Styan (1973). Browne (1974) was the first to present the relationship between the Hadamard and Kronecker product (square case). Faliva (1983) and Liu (1995) treated the rectangular case. See also Neudecker, Liu, and Polasek (1995); Neudecker, Polasek, and Liu (1995); and Neudecker and Liu (2001a,b) for a survey and applications of the Hadamard product in a random environment.

7. The commutation matrix was systematically studied by Magnus and Neudecker (1979). See also Magnus and Neudecker (1986). Theorem 3.10 is due to Neudecker and Wansbeek (1983). The matrix N_n was introduced by Browne (1974). For a rigorous and extensive treatment, see Magnus (1988).

8. See Browne (1974) and Magnus and Neudecker (1980, 1986) for further properties of the duplication matrix. Theorem 3.14 follows from Equations (60), (62), and (64) in Magnus and Neudecker (1986). A systematic treatment of linear structures (of which symmetry is one example) is given in Magnus (1988).

9‐10. See Holly and Magnus (1988).

11. See also Debreu (1952), Black and Morimoto (1968), and Farebrother (1977).

12. See also Chipman (1964).

13. See Pringle and Rayner (1971, Chapter 3), Rao (1973, Section 4i.1), and Magnus (1990).