Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

5.2 Orthogonal Subspaces

Let A be an $m \times n$ $m \times n$ matrix and let $x \in N (A)$ $x \in N (A)$ , the null space of A. Since $A x = 0$ $A x = 0$ , we have

a_{i 1} x_{1} + a_{i 2} x_{2} + … + a_{i n} x_{n} = 0

$a_{i 1} x_{1} + a_{i 2} x_{2} + … + a_{i n} x_{n} = 0$ (1)

for $i = 1, …, m$ $i = 1, …, m$ . Equation (1) says that x is orthogonal to the ith column vector of $A^{T}$ $A^{T}$ for $i = 1, …, m$ $i = 1, …, m$ . Since x is orthogonal to each column vector of $A^{T}$ $A^{T}$ , it is orthogonal to any linear combination of the column vectors of $A^{T}$ $A^{T}$ . So if y is any vector in the column space of $A^{T}$ $A^{T}$ , then $x^{T} y = 0$ $x^{T} y = 0$ . Thus, each vector in N(A) is orthogonal to every vector in the column space of $A^{T}$ $A^{T}$ . When two subspaces of $ℝ^{n}$ $ℝ^{n}$ have this property, we say that they are orthogonal.

Example 1

Let X be the subspace of $ℝ^{3}$ $ℝ^{3}$ spanned by $e_{1}$ $e_{1}$ , and let Y be the subspace spanned by $e_{2}$ $e_{2}$ . If $x \in X$ $x \in X$ , these vectors must be of the form

\begin{matrix} x = [\begin{matrix} x_{1} \\ 0 \\ 0 \end{matrix}] & \begin{matrix} and & y = [\begin{matrix} 0 \\ y_{2} \\ 0 \end{matrix}] \end{matrix} \end{matrix}

$\begin{matrix} x = [\begin{matrix} x_{1} \\ 0 \\ 0 \end{matrix}] & \begin{matrix} and & y = [\begin{matrix} 0 \\ y_{2} \\ 0 \end{matrix}] \end{matrix} \end{matrix}$

Thus,

x^{T} y = x_{1} \cdot 0 + 0 \cdot y_{2} + 0 \cdot 0 = 0

$x^{T} y = x_{1} \cdot 0 + 0 \cdot y_{2} + 0 \cdot 0 = 0$

Therefore, $X ⊥ Y$ $X ⊥ Y$ .

The concept of orthogonal subspaces does not always agree with our intuitive idea of perpendicularity. For example, the floor and wall of the classroom “look” orthogonal, but the xy-plane and the yz-plane are not orthogonal subspaces. Indeed, we can think of the vectors $x_{1} = {(1, 1, 0)}^{T}$ $x_{1} = {(1, 1, 0)}^{T}$ and $x_{2} = {(0, 1, 0)}^{T}$ $x_{2} = {(0, 1, 0)}^{T}$ as lying in the xy- and yz-planes, respectively. Since

x_{1}^{T} x_{2} = 1 \cdot 0 + 1 \cdot 1 + 0 \cdot 1 = 1

$x_{1}^{T} x_{2} = 1 \cdot 0 + 1 \cdot 1 + 0 \cdot 1 = 1$

the subspaces are not orthogonal. The next example shows that the subspace corresponding to the z-axis is orthogonal to the subspace corresponding to the xy-plane.

Example 2

Let X be the subspace of $ℝ^{3}$ $ℝ^{3}$ spanned by $e_{1}$ $e_{1}$ and $e_{2}$ $e_{2}$ , and let Y be the subspace spanned by $e_{3}$ $e_{3}$ . If $x \in X$ $x \in X$ and $y \in Y$ $y \in Y$ , then

x^{T} y = x_{1} \cdot 0 + x_{2} \cdot 0 + 0 \cdot y_{3} = 0

$x^{T} y = x_{1} \cdot 0 + x_{2} \cdot 0 + 0 \cdot y_{3} = 0$

Thus, $X ⊥ Y$ $X ⊥ Y$ . Furthermore, if z is any vector in $ℝ^{3}$ $ℝ^{3}$ that is orthogonal to every vector in Y, then $z ⊥ e_{3}$ $z ⊥ e_{3}$ , and hence

z_{3} = z^{T} e_{3} = 0

$z_{3} = z^{T} e_{3} = 0$

But if $z_{3} = 0$ $z_{3} = 0$ , then $z \in X$ $z \in X$ . Therefore, X is the set of all vectors in $ℝ^{3}$ $ℝ^{3}$ that are orthogonal to every vector in Y (see Figure 5.2.1).

A vector diagram represents orthogonality on a three dimensional plane.

Figure 5.2.1. Full Alternative Text

Note

The subspaces $X = Span (e_{1})$ $X = Span (e_{1})$ and $X = Span (e_{2})$ $X = Span (e_{2})$ of $ℝ^{3}$ $ℝ^{3}$ given in Example 1 are orthogonal, but they are not orthogonal complements. Indeed,

\begin{matrix} X^{⊥} = Span (e_{2}, e_{3}) & \begin{matrix} and & Y^{⊥} = Span (e_{1}, e_{3}) \end{matrix} \end{matrix}

$\begin{matrix} X^{⊥} = Span (e_{2}, e_{3}) & \begin{matrix} and & Y^{⊥} = Span (e_{1}, e_{3}) \end{matrix} \end{matrix}$

Remarks

1. If X and Y are orthogonal subspaces of $ℝ^{n}$ $ℝ^{n}$ , then $X \cap Y = {0}$ $X \cap Y = {0}$ .
2. If Y is a subspace of $ℝ^{n}$ $ℝ^{n}$ , then $Y^{⊥}$ $Y^{⊥}$ is also a subspace of $ℝ^{n}$ $ℝ^{n}$ .

Proof of (1)

If $x \in X \cap Y$ $x \in X \cap Y$ and $X ⊥ Y$ $X ⊥ Y$ , then ${‖ x ‖}^{2} = x^{T} x = 0$ ${‖ x ‖}^{2} = x^{T} x = 0$ and hence $x = 0$ $x = 0$ .

∎

Proof of (2)

If $x \in Y^{⊥}$ $x \in Y^{⊥}$ and α is a scalar, then for any $y \in Y$ $y \in Y$ ,

{(α x)}^{T} y = α (x^{T} y) = α \cdot 0 = 0

${(α x)}^{T} y = α (x^{T} y) = α \cdot 0 = 0$

Therefore, $α x \subset Y^{⊥}$ $α x \subset Y^{⊥}$ . If $x_{1}$ $x_{1}$ and $x_{2}$ $x_{2}$ are elements of $Y^{⊥}$ $Y^{⊥}$ , then

{(x_{1} + x_{2})}^{T} y = x_{1}^{T} y + x_{2}^{T} y = 0 + 0 = 0

${(x_{1} + x_{2})}^{T} y = x_{1}^{T} y + x_{2}^{T} y = 0 + 0 = 0$

for each $y \in Y$ $y \in Y$ . Hence, $x_{1} + x_{2} \in Y^{⊥}$ $x_{1} + x_{2} \in Y^{⊥}$ . Therefore, $Y^{⊥}$ $Y^{⊥}$ is a subspace of $ℝ^{n}$ $ℝ^{n}$ .

∎

Fundamental Subspaces

Let A be an $m \times n$ $m \times n$ matrix. We saw in Chapter 3 that a vector $b \in ℝ^{m}$ $b \in ℝ^{m}$ is in the column space of A if and only if $b = A x$ $b = A x$ for some $x \in ℝ^{n}$ $x \in ℝ^{n}$ . If we think of A as a linear transformation mapping $ℝ^{n}$ $ℝ^{n}$ into $ℝ^{m}$ $ℝ^{m}$ , then the column space of A is the same as the range of A. Let us denote the range of A by R(A). Thus,

\begin{matrix} R (A) & = {b \in ℝ^{m} | b = A x for some x \in ℝ^{n}} \\ = the column space of A \end{matrix}

$\begin{matrix} R (A) & = {b \in ℝ^{m} | b = A x for some x \in ℝ^{n}} \\ = the column space of A \end{matrix}$

The column space of $A^{T}, R {(A)}^{T}$ $A^{T}, R {(A)}^{T}$ ), is a subspace of $ℝ^{n}$ $ℝ^{n}$ :

R (A^{T}) = {y \in ℝ^{n} | y = A^{T} x for some x \in ℝ^{m}}

$R (A^{T}) = {y \in ℝ^{n} | y = A^{T} x for some x \in ℝ^{m}}$

The column space of $R (A^{T})$ $R (A^{T})$ is essentially the same as the row space of A, except that it consists of vectors in $ℝ^{n}$ $ℝ^{n}$ ( $n \times 1$ $n \times 1$ matrices) rather than n-tuples. Thus, $y \in R (A^{T})$ $y \in R (A^{T})$ ) if and only if $y^{T}$ $y^{T}$ is in the row space of A. We have seen that $R (A^{T}) ⊥ N (A)$ $R (A^{T}) ⊥ N (A)$ . The following theorem shows that N(A) is actually the orthogonal complement of $R (A^{T})$ $R (A^{T})$ .

Theorem 5.2.1 Fundamental Subspaces Theorem

If A is an $m \times n$ $m \times n$ matrix, then $N (A) = R {(A^{T})}^{⊥} a n d N (A^{T}) = R {(A)}^{⊥}$ $N (A) = R {(A^{T})}^{⊥} a n d N (A^{T}) = R {(A)}^{⊥}$ .

Proof

On the one hand, we have already seen that $N (A) ⊥ R (A^{T})$ $N (A) ⊥ R (A^{T})$ , and this implies that $N (A) \subset R {(A^{T})}^{⊥}$ $N (A) \subset R {(A^{T})}^{⊥}$ . On the other hand, if x is any vector in $R {(A^{T})}^{⊥}$ $R {(A^{T})}^{⊥}$ , then x is orthogonal to each of the column vectors of $A^{T}$ $A^{T}$ and, consequently, $A x = 0$ $A x = 0$ . Thus, x must be an element of N(A) and hence $N (A) = R {(A^{T})}^{⊥}$ $N (A) = R {(A^{T})}^{⊥}$ . This proof does not depend on the dimensions of A. In particular, the result will also hold for the matrix $B = A^{T}$ $B = A^{T}$ . Consequently,

N (A^{T}) = N (B) = R {(B^{T})}^{⊥} = R {(A)}^{⊥}

$N (A^{T}) = N (B) = R {(B^{T})}^{⊥} = R {(A)}^{⊥}$

∎

Example 3

Let

A = [\begin{matrix} 1 & 0 \\ 2 & 0 \end{matrix}]

$A = [\begin{matrix} 1 & 0 \\ 2 & 0 \end{matrix}]$

The column space of A consists of all vectors of the form

[\begin{matrix} α \\ 2 α \end{matrix}] = α [\begin{matrix} 1 \\ 2 \end{matrix}]

$[\begin{matrix} α \\ 2 α \end{matrix}] = α [\begin{matrix} 1 \\ 2 \end{matrix}]$

Note that if x is any vector in $ℝ^{2}$ $ℝ^{2}$ and $b = A x$ $b = A x$ , then

b = [\begin{matrix} 1 & 0 \\ 2 & 0 \end{matrix}] [\begin{matrix} x_{1} \\ x_{2} \end{matrix}] = [\begin{matrix} 1 x_{1} \\ 2 x_{1} \end{matrix}] = x_{1} [\begin{matrix} 1 \\ 2 \end{matrix}]

$b = [\begin{matrix} 1 & 0 \\ 2 & 0 \end{matrix}] [\begin{matrix} x_{1} \\ x_{2} \end{matrix}] = [\begin{matrix} 1 x_{1} \\ 2 x_{1} \end{matrix}] = x_{1} [\begin{matrix} 1 \\ 2 \end{matrix}]$

The null space of $A^{T}$ $A^{T}$ consists of all vectors of the form $β (- 2, 1)$ $β (- 2, 1)$ . Since ${(1, 2)}^{T}$ ${(1, 2)}^{T}$ and ${(- 2, 1)}^{T}$ ${(- 2, 1)}^{T}$ are orthogonal, it follows that every vector in R(A) will be orthogonal to every vector in $N (A^{T})$ $N (A^{T})$ . The same relationship holds between $R (A^{T})$ $R (A^{T})$ and N(A). $R (A^{T})$ $R (A^{T})$ consists of vectors of the form $α e_{1}$ $α e_{1}$ , and N(A) consists of all vectors of the form $β e_{2}$ $β e_{2}$ . Since $e_{1}$ $e_{1}$ and $e_{2}$ $e_{2}$ are orthogonal, it follows that each vector in $R (A^{T})$ $R (A^{T})$ is orthogonal to every vector in N(A).

Theorem 5.2.1 is one of the most important theorems in this chapter. In Section 5.3, we will see that the result $N (A^{T}) = R {(A)}^{⊥}$ $N (A^{T}) = R {(A)}^{⊥}$ provides a key to solving least squares problems. For the present, we will use Theorem 5.2.1 to prove the following theorem, which, in turn, will be used to establish two more important results about orthogonal subspaces.

Theorem 5.2.2

If S is a subspace of $ℝ^{n}$ $ℝ^{n}$ , then $S + dim S^{⊥} = n$ $S + dim S^{⊥} = n$ . Furthermore, if ${x_{1}, …, x_{r}}$ ${x_{1}, …, x_{r}}$ is a basis for S and ${x_{r + 1}, …, x_{n}}$ ${x_{r + 1}, …, x_{n}}$ is a basis for $S^{⊥}$ $S^{⊥}$ , then ${x_{1}, …, x_{r}, x_{r + 1}, …, x_{n}}$ ${x_{1}, …, x_{r}, x_{r + 1}, …, x_{n}}$ is a basis for $ℝ^{n}$ $ℝ^{n}$ .

Proof

If $S = {0}$ $S = {0}$ , then $S^{⊥} = ℝ^{n}$ $S^{⊥} = ℝ^{n}$ and

dim S + dim S^{⊥} = 0 + n = n

$dim S + dim S^{⊥} = 0 + n = n$

If $S \neq {0}$ $S \neq {0}$ , then let ${x_{1}, …, x_{r}}$ ${x_{1}, …, x_{r}}$ , be a basis for S and define X to be an $r \times n$ $r \times n$ matrix whose ith row is $x_{i}^{T}$ $x_{i}^{T}$ for each i. By construction, the matrix X has rank r and $R (X^{T}) = S$ $R (X^{T}) = S$ . By Theorem 5.2.1,

S^{⊥} = R {(X^{T})}^{⊥} = N (X)

$S^{⊥} = R {(X^{T})}^{⊥} = N (X)$

It follows from Theorem 3.6.5 that

\dim S^{⊥} = \dim N (X) = n - r

$\dim S^{⊥} = \dim N (X) = n - r$

To show that ${x_{1}, …, x_{r}, x_{r + 1}, …, x_{n}}$ ${x_{1}, …, x_{r}, x_{r + 1}, …, x_{n}}$ is a basis for $ℝ^{n}$ $ℝ^{n}$ , it suffices to show that the n vectors are linearly independent. Suppose that

c_{1} x_{1} + … + c_{r} x_{r} + c_{r + 1} x_{r} + c_{r + 1} x_{r + 1} + … + c_{n} x_{n} = 0

$c_{1} x_{1} + … + c_{r} x_{r} + c_{r + 1} x_{r} + c_{r + 1} x_{r + 1} + … + c_{n} x_{n} = 0$

Let $y = c_{1} x_{1} + … + c_{r} x_{r}$ $y = c_{1} x_{1} + … + c_{r} x_{r}$ and $z = c_{r + 1} x_{r + 1} + … + c_{n} x_{n}$ $z = c_{r + 1} x_{r + 1} + … + c_{n} x_{n}$ . We then have

\begin{matrix} y + z & = 0 \\ y & = - z \end{matrix}

$\begin{matrix} y + z & = 0 \\ y & = - z \end{matrix}$

Thus, y and z are both elements of $S \cap S^{⊥}$ $S \cap S^{⊥}$ . But $S \cap S^{⊥} = {0}$ $S \cap S^{⊥} = {0}$ . Therefore,

\begin{matrix} c_{1} x_{1} + … + c_{r} x_{r} = 0 \\ c_{r + 1} x_{r + 1} + … + c_{n} x_{n} = 0 \end{matrix}

$\begin{matrix} c_{1} x_{1} + … + c_{r} x_{r} = 0 \\ c_{r + 1} x_{r + 1} + … + c_{n} x_{n} = 0 \end{matrix}$

Since $x_{1}, …, x_{r}$ $x_{1}, …, x_{r}$ are linearly independent,

c_{1} = c_{2} = \dots = c_{r} = 0

$c_{1} = c_{2} = \dots = c_{r} = 0$

Similarly, $x_{r + 1}, …, x_{n}$ $x_{r + 1}, …, x_{n}$ are linearly independent and hence

c_{r + 1} = c_{r + 2} = \dots = c_{n} = 0

$c_{r + 1} = c_{r + 2} = \dots = c_{n} = 0$

So $x_{1}, x_{2}, …, x_{n}$ $x_{1}, x_{2}, …, x_{n}$ are linearly independent and form a basis for $ℝ^{n}$ $ℝ^{n}$ .

∎

Given a subspace S of $ℝ^{n}$ $ℝ^{n}$ , we will use Theorem 5.2.2 to prove that each $x ∊ ℝ^{n}$ $x ∊ ℝ^{n}$ can be expressed uniquely as a sum $y + z$ $y + z$ , where $y \in S$ $y \in S$ and $z \in S^{⊥}$ $z \in S^{⊥}$ .

Theorem 5.2.3

If S is a subspace of $ℝ^{n}$ $ℝ^{n}$ , then

ℝ^{n} = S \oplus S^{⊥}

$ℝ^{n} = S \oplus S^{⊥}$

Proof

The result is trivial if either $S = {0}$ $S = {0}$ or $S = ℝ^{n}$ $S = ℝ^{n}$ . In the case where dim $S = r, 0 < r < n$ $S = r, 0 < r < n$ , it follows from Theorem 5.2.2 that each vector $x \in ℝ^{n}$ $x \in ℝ^{n}$ can be represented in the form

x = c_{1} x_{1} + … + c_{r} x_{r} + c_{r} x_{r + 1} + … + c_{n} x_{n}

$x = c_{1} x_{1} + … + c_{r} x_{r} + c_{r} x_{r + 1} + … + c_{n} x_{n}$

where ${x_{1}, …, x_{r}}$ ${x_{1}, …, x_{r}}$ is a basis for S and ${x_{r + 1}, …, x_{n}}$ ${x_{r + 1}, …, x_{n}}$ is a basis for $S^{⊥}$ $S^{⊥}$ . If we let

\begin{matrix} u = c_{1} x_{1} + … + c_{r} x_{r} & \begin{matrix} and & v = c_{r + 1} x_{r + 1} + … + c_{n} x_{n} \end{matrix} \end{matrix}

$\begin{matrix} u = c_{1} x_{1} + … + c_{r} x_{r} & \begin{matrix} and & v = c_{r + 1} x_{r + 1} + … + c_{n} x_{n} \end{matrix} \end{matrix}$

then $u \in S, v \in S^{⊥}$ $u \in S, v \in S^{⊥}$ , and $x = u + v$ $x = u + v$ . To show uniqueness, suppose that x can also be written as a sum $y + z$ $y + z$ , where $y ∊ S$ $y ∊ S$ and $z ∊ S^{⊥}$ $z ∊ S^{⊥}$ . Thus,

\begin{matrix} u + v & \begin{matrix} \begin{matrix} = \end{matrix} & x = y + z \end{matrix} \\ u - v & \begin{matrix} \begin{matrix} = \end{matrix} & z - v \end{matrix} \end{matrix}

$\begin{matrix} u + v & \begin{matrix} \begin{matrix} = \end{matrix} & x = y + z \end{matrix} \\ u - v & \begin{matrix} \begin{matrix} = \end{matrix} & z - v \end{matrix} \end{matrix}$

But $u - y \in S$ $u - y \in S$ and ${z - v \in S}^{⊥}$ ${z - v \in S}^{⊥}$ , so each is in $S \cap S^{⊥}$ $S \cap S^{⊥}$ . Since

S \cap S^{⊥} = {0}

$S \cap S^{⊥} = {0}$

it follows that

\begin{matrix} u = y & \begin{matrix} and & v = z \end{matrix} \end{matrix}

$\begin{matrix} u = y & \begin{matrix} and & v = z \end{matrix} \end{matrix}$

∎

Theorem 5.2.4

If S is a subspace of $ℝ^{n}$ $ℝ^{n}$ , then ${(S^{⊥})}^{⊥} = S$ ${(S^{⊥})}^{⊥} = S$ .

Proof

On the one hand, if $x \in S$ $x \in S$ , then x is orthogonal to each y in $S^{⊥}$ $S^{⊥}$ . Therefore, $x \in {(S^{⊥})}^{⊥}$ $x \in {(S^{⊥})}^{⊥}$ and hence $S \subset {(S^{⊥})}^{⊥}$ $S \subset {(S^{⊥})}^{⊥}$ . On the other hand, suppose that z is an arbitrary element of ${(S^{⊥})}^{⊥}$ ${(S^{⊥})}^{⊥}$ . By Theorem 5.2.3, we can write z as a sum $u + v$ $u + v$ , where $u \in S$ $u \in S$ and $v \in S^{⊥}$ $v \in S^{⊥}$ . Since $v \in S^{⊥}$ $v \in S^{⊥}$ , it is orthogonal to both u and z. It then follows that

0 = v^{T} z = v^{T} u + v^{T} v = v^{T} v

$0 = v^{T} z = v^{T} u + v^{T} v = v^{T} v$

and, consequently, $v = 0$ $v = 0$ . Therefore, $z = u \in S$ $z = u \in S$ and hence $S = {(S^{⊥})}^{⊥}$ $S = {(S^{⊥})}^{⊥}$ .

∎

It follows from Theorem 5.2.4 that if T is the orthogonal complement of a subspace S, then S is the orthogonal complement of T, and we may say simply that S and T are orthogonal complements. In particular, it follows from Theorem 5.2.1 that N(A) and $R (A^{T})$ $R (A^{T})$ are orthogonal complements of each other and that $N (A^{T})$ $N (A^{T})$ and R(A) are orthogonal complements. Hence, we may write

{\begin{matrix} N {(A)}^{⊥} = R (A^{T}) & \begin{matrix} and & N (A^{T}) \end{matrix} \end{matrix}}^{⊥} = R (A)

${\begin{matrix} N {(A)}^{⊥} = R (A^{T}) & \begin{matrix} and & N (A^{T}) \end{matrix} \end{matrix}}^{⊥} = R (A)$

Recall that the system $A x = b$ $A x = b$ is consistent if and only if $b \in R (A)$ $b \in R (A)$ . Since $R (A) = N {(A^{T})}^{⊥}$ $R (A) = N {(A^{T})}^{⊥}$ , we have the following result, which may be considered a corollary to Theorem 5.2.1.

Corollary 5.2.5

If A is an $m \times n$ $m \times n$ matrix and $b \in ℝ^{m}$ $b \in ℝ^{m}$ , then either there is a vector $x \in ℝ^{n}$ $x \in ℝ^{n}$ such that $A x = b$ $A x = b$ or there is a vector $y \in ℝ^{m}$ $y \in ℝ^{m}$ such that $A^{T} y = 0$ $A^{T} y = 0$ and $y^{T} b \neq 0$ $y^{T} b \neq 0$ .

Corollary 5.2.5 is illustrated in Figure 5.2.2 for the case where R(A) is a two-dimensional subspace of $ℝ^{3}$ $ℝ^{3}$ . The angle $θ$ $θ$ in the figure will be a right angle if and only if $b \in R (A)$ $b \in R (A)$ .

A vector diagram has two vectors on a plane.

Figure 5.2.2. Full Alternative Text

Example 4

Let

A = [\begin{matrix} 1 & 1 & 2 \\ 0 & 1 & 1 \\ 1 & 3 & 4 \end{matrix}]

$A = [\begin{matrix} 1 & 1 & 2 \\ 0 & 1 & 1 \\ 1 & 3 & 4 \end{matrix}]$

Find the bases for N(A), $R (A^{T}), N (A^{T})$ $R (A^{T}), N (A^{T})$ , and R(A).

SOLUTION

We can find bases for N(A) and $R (A^{T})$ $R (A^{T})$ by transforming A into reduced row echelon form:

[\begin{matrix} 1 & 1 & 2 \\ 0 & 1 & 1 \\ 1 & 3 & 4 \end{matrix}] \to [\begin{matrix} 1 & 1 & 2 \\ 0 & 1 & 1 \\ 0 & 2 & 2 \end{matrix}] \to [\begin{matrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{matrix}]

$[\begin{matrix} 1 & 1 & 2 \\ 0 & 1 & 1 \\ 1 & 3 & 4 \end{matrix}] \to [\begin{matrix} 1 & 1 & 2 \\ 0 & 1 & 1 \\ 0 & 2 & 2 \end{matrix}] \to [\begin{matrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{matrix}]$

Since (1, 0, 1) and (0, 1, 1) form a basis for the row space of A, it follows that ${(1, 0, 1)}^{T}$ ${(1, 0, 1)}^{T}$ and ${(0, 1, 1)}^{T}$ ${(0, 1, 1)}^{T}$ form a basis for $R (A^{T})$ $R (A^{T})$ . If $x \in N (A)$ $x \in N (A)$ , it follows from the reduced row echelon form of A that

\begin{matrix} x_{1} + x_{3} & \begin{matrix} \begin{matrix} = \end{matrix} & 0 \end{matrix} \\ x_{2} + x_{3} & \begin{matrix} \begin{matrix} = \end{matrix} & 0 \end{matrix} \end{matrix}

$\begin{matrix} x_{1} + x_{3} & \begin{matrix} \begin{matrix} = \end{matrix} & 0 \end{matrix} \\ x_{2} + x_{3} & \begin{matrix} \begin{matrix} = \end{matrix} & 0 \end{matrix} \end{matrix}$

Thus,

x_{1} = x_{2} = - x_{3}

$x_{1} = x_{2} = - x_{3}$

Setting $x_{3} = α$ $x_{3} = α$ , we see that N(A) consists of all vectors of the form $α {(- 1, - 1, 1)}^{T}$ $α {(- 1, - 1, 1)}^{T}$ . Note that ${(- 1, - 1, 1)}^{T}$ ${(- 1, - 1, 1)}^{T}$ is orthogonal to ${(1, 0, 1)}^{T}$ ${(1, 0, 1)}^{T}$ and ${(0, 1, 1)}^{T}$ ${(0, 1, 1)}^{T}$ .

To find bases for R(A) and $N (A^{T})$ $N (A^{T})$ , transform $A^{T}$ $A^{T}$ to reduced row echelon form.

[\begin{matrix} 1 & 0 & 1 \\ 1 & 1 & 3 \\ 2 & 1 & 4 \end{matrix}] \to [\begin{matrix} 1 & 0 & 1 \\ 0 & 1 & 2 \\ 0 & 1 & 2 \end{matrix}] \to [\begin{matrix} 1 & 0 & 1 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{matrix}]

$[\begin{matrix} 1 & 0 & 1 \\ 1 & 1 & 3 \\ 2 & 1 & 4 \end{matrix}] \to [\begin{matrix} 1 & 0 & 1 \\ 0 & 1 & 2 \\ 0 & 1 & 2 \end{matrix}] \to [\begin{matrix} 1 & 0 & 1 \\ 0 & 1 & 2 \\ 0 & 0 & 0 \end{matrix}]$

Thus, ${(1, 0, 1)}^{T}$ ${(1, 0, 1)}^{T}$ and ${(0, 1, 2)}^{T}$ ${(0, 1, 2)}^{T}$ form a basis for R(A). If $x \in N (A^{T})$ $x \in N (A^{T})$ , then $x_{1} = - x_{3}, x_{2} = - 2 x_{3}$ $x_{1} = - x_{3}, x_{2} = - 2 x_{3}$ . Hence, $N (A^{T})$ $N (A^{T})$ is the subspace of $ℝ^{3}$ $ℝ^{3}$ spanned by ${(- 1, - 2, 1)}^{T}$ ${(- 1, - 2, 1)}^{T}$ . Note that ${(- 1, - 2, 1)}^{T}$ ${(- 1, - 2, 1)}^{T}$ is orthogonal to ${(1, 0, 1)}^{T}$ ${(1, 0, 1)}^{T}$ and ${(0, 1, 2)}^{T}$ ${(0, 1, 2)}^{T}$ .

We saw in Chapter 3 that the row space and the column space have the same dimension. If A has rank r, then

dim R (A) = dim R (A^{T}) = r

$dim R (A) = dim R (A^{T}) = r$

Actually, A can be used to establish a one-to-one correspondence between $R (A^{T})$ $R (A^{T})$ and R(A).

We can think of an $m \times n$ $m \times n$ matrix A as a linear transformation from $ℝ^{n}$ $ℝ^{n}$ to $ℝ^{m}$ $ℝ^{m}$ :

x \in ℝ^{n} \to A x \in ℝ^{m}

$x \in ℝ^{n} \to A x \in ℝ^{m}$

Since $R (A^{T})$ $R (A^{T})$ and N(A) are orthogonal complements in $ℝ^{n}$ $ℝ^{n}$ ,

ℝ^{n} = R (A^{T}) \oplus N (A)

$ℝ^{n} = R (A^{T}) \oplus N (A)$

Each vector $x \in ℝ^{n}$ $x \in ℝ^{n}$ can be written as a sum

\begin{matrix} x = y + z, & \begin{matrix} \begin{matrix} y \in R (A^{T}), \end{matrix} & z \in N (A) \end{matrix} \end{matrix}

$\begin{matrix} x = y + z, & \begin{matrix} \begin{matrix} y \in R (A^{T}), \end{matrix} & z \in N (A) \end{matrix} \end{matrix}$

It follows that

\begin{matrix} A x = A y + A z = A y & for each x \in ℝ^{n} \end{matrix}

$\begin{matrix} A x = A y + A z = A y & for each x \in ℝ^{n} \end{matrix}$

and hence

R (A) = {A x | x ℝ^{n}} = {A y | y \in R (A^{T}}

$R (A) = {A x | x ℝ^{n}} = {A y | y \in R (A^{T}}$

Thus, if we restrict the domain of A to $R (A^{T})$ $R (A^{T})$ , then A maps $R (A^{T})$ $R (A^{T})$ onto R(A). Furthermore, the mapping is one-to-one. Indeed, if $x_{1}, x_{2} \in R (A^{T})$ $x_{1}, x_{2} \in R (A^{T})$ and

A x_{1} = A x_{2}

$A x_{1} = A x_{2}$

then

A (x_{1} - x_{2}) = 0

$A (x_{1} - x_{2}) = 0$

and hence

x_{1} - x_{2} \in R (A^{T}) \cap N (A)

$x_{1} - x_{2} \in R (A^{T}) \cap N (A)$

Since $R (A^{T}) \cap N (A) = {0}$ $R (A^{T}) \cap N (A) = {0}$ , it follows that $x_{1} = x_{2}$ $x_{1} = x_{2}$ . Therefore, we can think of A as determining a one-to-one correspondence between $R (A^{T})$ $R (A^{T})$ and R(A). Since each $b \in R (A)$ $b \in R (A)$ corresponds to exactly one $y \in R (A^{T})$ $y \in R (A^{T})$ , we can define an inverse transformation from R(A) to $R (A^{T})$ $R (A^{T})$ . Indeed, every $m \times n$ $m \times n$ matrix A is invertible when viewed as a linear transformation from $R (A^{T})$ $R (A^{T})$ to R(A).

Example 5

Let $A = [\begin{matrix} 2 & 0 & 0 \\ 0 & 3 & 0 \end{matrix}] . R (A^{T})$ $A = [\begin{matrix} 2 & 0 & 0 \\ 0 & 3 & 0 \end{matrix}] . R (A^{T})$ is spanned by $e_{1}$ $e_{1}$ and $e_{2}$ $e_{2}$ , and N(A) is spanned by $e_{3}$ $e_{3}$ . Any vector $x \in ℝ^{3}$ $x \in ℝ^{3}$ can be written as a sum

x = y + z

$x = y + z$

where

\begin{matrix} y = {(x_{1}, x_{2}, 0)}^{T} \in R (A^{T}) & \begin{matrix} and & z = (0, 0, x_{3})^{T} \in N (A) \end{matrix} \end{matrix}

$\begin{matrix} y = {(x_{1}, x_{2}, 0)}^{T} \in R (A^{T}) & \begin{matrix} and & z = (0, 0, x_{3})^{T} \in N (A) \end{matrix} \end{matrix}$

If we restrict ourselves to vectors $y \in R (A^{T})$ $y \in R (A^{T})$ , then

y = [\begin{matrix} x_{1} \\ x_{2} \\ 0 \end{matrix}] \to A y = [\begin{matrix} 2 x_{1} \\ 3 x_{2} \end{matrix}]

$y = [\begin{matrix} x_{1} \\ x_{2} \\ 0 \end{matrix}] \to A y = [\begin{matrix} 2 x_{1} \\ 3 x_{2} \end{matrix}]$

In this case, $R (A) = ℝ^{2}$ $R (A) = ℝ^{2}$ and the inverse transformation from R(A) to $R (A^{T})$ $R (A^{T})$ is defined by

b = [\begin{matrix} b_{1} \\ b_{2} \end{matrix}] \to [\begin{matrix} \frac{1}{2} b_{1} \\ \frac{1}{3} b_{2} \\ 0 \end{matrix}]

$b = [\begin{matrix} b_{1} \\ b_{2} \end{matrix}] \to [\begin{matrix} \frac{1}{2} b_{1} \\ \frac{1}{3} b_{2} \\ 0 \end{matrix}]$

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for 5.2 Orthogonal Subspaces

Create new playlist

Sign In

Sign Up

Table of Contents for
5.2 Orthogonal Subspaces