Chapter 14

General Linear Spaces

In Sections 4.3 and 9.2, we had a first look at the concept of linear spaces, also called vector spaces, by examining the properties of the spaces of all 2D and 3D vectors. In this chapter, we will provide a framework for linear spaces that are not only of dimension two or three, but of possibly much higher dimension. These spaces tend to be somewhat abstract, but they are a powerful concept in dealing with many real-life problems, such as car crash simulations, weather forecasts, or computer games. The linear space of cubic polynomials over [0, 1] is important for many applications. Figure 14.1 provides an artistic illustration of some of the elements of this linear space. Hence the term “general” in the chapter title refers to the dimension and abstraction that we will study.

14.1 Basic Properties of Linear Spaces

We denote a linear space of dimension n by $L_n$. The elements of $L_n$ are vectors, denoted (as before) by boldface letters such as u or v. We need two operations to be defined on the elements of $L_n$: addition and multiplication by a scalar such as s or t. With these operations in place, the defining property for a linear space is that any linear combination of vectors,

$\begin{array}{rr}\hfill w=su+tv,& \hfill \left(14.1\right)\end{array}$

results in a vector in the same space. This is called the linearity property. Note that both s and t may be zero, asserting that every linear space has a zero vector in it. This equation is familiar to us by now, as we first encountered it in Section 2.6, and linear combinations have been central to working in the linear spaces of ℝ2 and ℝ3.1

But in this chapter, we want to generalize linear spaces; we want to include new kinds of vectors. For example, our linear space could be the quadratic polynomials or all 3 × 3matrices. Because the objects in the linear space are not always in the format of what we traditionally call a vector, we choose to call these linear spaces, which focuses on the linearity property rather than vector spaces. Both terms are accepted. Thus, we will expand our notation for objects of a linear space—we will not always use boldface vector notation. In this section, we will look at just a few examples of linear spaces, more examples of linear spaces will appear in Section 14.5.

Example 14.1

Let’s start with a very familiar example of a linear space: ℝ2. Suppose

$\begin{array}{rrr}\hfill u=\left[\begin{array}{r}\hfill 1\\ \hfill 1\end{array}\right]& \hfill \text{and}& \hfill v=\left[\begin{array}{c}-2\\ 3\end{array}\right]\end{array}$

are elements of this space; we know that

$w=2u+v=\left[\begin{array}{c}0\\ 5\end{array}\right]$

is also in ℝ2.

Example 14.2

Let the linear space $M_{2\times 2}$ be the set of all 2 × 2 matrices. We know that the linearity property (14.1) holds because of the rules of matrix arithmetic from Section 4.12.

Example 14.3

Let’s define ${\mathit{V}}_2$ to be all vectors w in ℝ2 that satisfy w2 ≥ 0. For example, e1 and e2 live in ${\mathit{V}}_2$. Is this a linear space?

It is not: we can form a linear combination as in (14.1) and produce

$v=0\times e_1+-1\times e_2=\left[\begin{array}{c}0\\ -1\end{array}\right],$

which is not in ${\mathit{V}}_2$.

In a linear space $L_n$ let’s look at a set of r vectors, where 1 ≤ r ≤ n. A set of vectors v1,...,vr is called linearly independent if it is impossible to express one of them as a linear combination of the others. For example, the equation

$v_1=s_2{v}_2+s_3{v}_3+...+s_r{v}_r$

will not have a solution set s2,..., sr in case the vectors v1,...,vr are linearly independent. As a simple consequence, the zero vector can be expressed only in a trivial manner in terms of linearly independent vectors, namely if

$0=s_1{v}_1+...+s_{r1}{v}_r$

then s1 = ... = sr = 0. If the zero vector can be expressed as a nontrivial combination of r vectors, then we say these vectors are linearly dependent.

If the vectors v1,...,vr are linearly independent, then the set of all vectors that may be expressed as a linear combination of them forms a subspace of $L_n$, of dimension r. We also say this subspace is spanned by v1, ..., vr. If this subspace equals the whole space $L_n$, then we call v1, ..., vn a basis for $L_n$, If $L_n$ is a linear space of dimension n, then any n +1 vectors in it are linearly dependent.

Example 14.4

Let’s consider one very familiar linear space: ℝ3. A commonly used basis forℝ3 are the linearly independent vectors e1, e2, e3. A linear combination of these basis vectors, for example,

$\begin{array}{l}\begin{array}{cccc}v=\left[\begin{array}{c}3\\ 4\\ 7\end{array}\right]& =3\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right]& +4\left[\begin{array}{c}0\\ 1\\ 0\end{array}\right]& +7\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right]\end{array}\\ \end{array}$

is also in ℝ3. Then the four vectors v e1, e2, e3 are linearly dependent.

Any one of the four vectors forms a one-dimensional subspace of ℝ3. Since v and each of the ei are linearly independent, any two vectors here form a two-dimensional subspace of ℝ3.

Example 14.5

In a 4D space ℝ4, let three vectors be given by

$\begin{array}{ccc}{v}_1=\left[\begin{array}{r}\hfill -1\\ \hfill 0\\ \hfill 0\\ \hfill 1\end{array}\right],& v_2=\left[\begin{array}{r}\hfill 5\\ \hfill 0\\ \hfill -3\\ \hfill 1\end{array}\right],& v_3=\left[\begin{array}{r}\hfill 3\\ \hfill 0\\ \hfill -3\\ \hfill 0\end{array}\right].\end{array}$

These three vectors are linearly dependent since

$\begin{array}{rrr}\hfill v_2=v_1+2v_3& \hfill \text{or}& \hfill 0=v_1-v_2+2v_3\end{array}.$

Our set {v1, v2, v3} contains only two linearly independent vectors, hence any two of them spans a subspace of ℝ4 of dimension two.

Example 14.6

In a 3D space ℝ3, let four vectors be given by

$\begin{array}{rrrr}\hfill v_1=\left[\begin{array}{r}\hfill -1\\ \hfill 0\\ \hfill 0\end{array}\right],& \hfill v_2=\left[\begin{array}{r}\hfill 1\\ \hfill 2\\ \hfill 0\end{array}\right],& \hfill v_3=\left[\begin{array}{r}\hfill 1\\ \hfill 2\\ \hfill -3\end{array}\right],& \hfill v_4=\left[\begin{array}{r}\hfill 0\\ \hfill 0\\ \hfill -3\end{array}\right].\end{array}$

These four vectors are linearly dependent since

$v_3=-v_1+2v_2+v_4.$

However, any set of three of these vectors is a basis for ℝ3.

14.2 Linear Maps

The linear map A that transforms the linear space $L_n$ to the linear space $L_m$, written as A : $L_n\to L_m$, preserves linear relationships. Let three preimage vectors v1, v2, v3 in $L_n$ be mapped to three image vectors Av1, Av2, Av3 in $L_m$. If there is a linear relationship among the preimages, vi, then the same relationship will hold for the images:

$\begin{array}{rrrr}\hfill v_1=\alpha v_2+\beta v_3& \hfill \text{implies}& \hfill A{v}_1=\alpha A{v}_2+\beta A{v}_3.& \hfill \left(14.2\right)\end{array}$

Maps that do not have this property are called nonlinear maps and are much harder to deal with.

Linear maps are conveniently written in terms of matrices. A vector v in $L_n$ is mapped to v′ in $L_m$,

$v^{\prime }=Av,$

where A has m rows and n columns. The matrix A describes the map from the [e1, ..., en]-system to the [a1, ..., an]-system, where the ai are in $L_m$ This means that v' is a linear combination v of the ai,

$v^{\prime }=v_1{a}_1+v_2{a}_2+...v_n{a}_{n,}$

and therefore it is in the column space of A.

Example 14.7

Suppose we have a map A : ℝ2 → ℝ3, defined by

$A=\left[\begin{array}{rr}\hfill 1& \hfill 0\\ \hfill 0& \hfill 1\\ \hfill 2& \hfill 2\end{array}\right].$

And suppose we have three vectors in ℝ2,

$\begin{array}{rrr}\hfill v_1=\left[\begin{array}{r}\hfill 1\\ \hfill 0\end{array}\right],& \hfill v_2=\left[\begin{array}{r}\hfill 0\\ \hfill 1\end{array}\right],& \hfill v_3=\left[\begin{array}{r}\hfill 2\\ \hfill 1\end{array}\right]\end{array},$

which are mapped to

$\begin{array}{rrr}\hfill {\widehat{v}}_1=\left[\begin{array}{r}\hfill 1\\ \hfill 0\\ \hfill 2\end{array}\right],& \hfill {\widehat{v}}_2=\left[\begin{array}{r}\hfill 0\\ \hfill 1\\ \hfill 2\end{array}\right],& \hfill {\widehat{v}}_3=\left[\begin{array}{r}\hfill 2\\ \hfill 1\\ \hfill 6\end{array}\right],\end{array}$

in ℝ3 by A.

The vi are linearly dependent since v3 = 2v1+ v2. This same linear combination holds for the ${\widehat{v}}_i$, asserting that the map preserves linear relationships.

The matrix A has a certain rank k—how can we infer this rank from the matrix? First of all, a matrix of size m × n, can be at most of rank k = min{m, n}. This is called full rank. In other words, a linear map can never increase dimension. It is possible to map $L_n$ to a higher dimensional space $L_m$. However, the images of $L_n$’s n basis vectors will span a subspace of $L_m$ of dimension at most n. Example 14.7 demonstrates this idea: the matrix A has rank 2, thus the ${\widehat{v}}_i$ live in a dimension 2 subspace of ℝ3. A matrix with rank less than this min {m, n} is called rank deficient. We perform forward elimination (possibly with row exchanges) until the matrix is in upper triangular form. If after forward elimination there are k nonzero rows, then the rank of A is k. This is equivalent to our definition in Section 4.2 that the rank is equal to the number of linearly independent column vectors. Figure 14.2 gives an illustration of some possible scenarios.

Example 14.8

Let us determine the rank of the matrix

$\left[\begin{array}{rrr}\hfill 1& \hfill 3& \hfill 4\\ \hfill 0& \hfill 1& \hfill 2\\ \hfill 1& \hfill 2& \hfill 2\\ \hfill -1& \hfill 1& \hfill 1\end{array}\right].$

We perform forward elimination to obtain

$\left[\begin{array}{rrr}\hfill 1& \hfill 3& \hfill 4\\ \hfill 0& \hfill 1& \hfill 2\\ \hfill 0& \hfill 0& \hfill -3\\ \hfill 0& \hfill 0& \hfill 0\end{array}\right].$

There is one row of zeroes, and we conclude that the matrix has rank 3, which is full rank since min {4, 3} = 3.

Next, let us take the matrix

$\left[\begin{array}{rrr}\hfill 1& \hfill 3& \hfill 4\\ \hfill 0& \hfill 1& \hfill 2\\ \hfill 1& \hfill 2& \hfill 2\\ \hfill 0& \hfill 1& \hfill 2\end{array}\right].$

Forward elimination yields

$\left[\begin{array}{rrr}\hfill 1& \hfill 3& \hfill 4\\ \hfill 0& \hfill 1& \hfill 2\\ \hfill 0& \hfill 0& \hfill 0\\ \hfill 0& \hfill 0& \hfill 0\end{array}\right],$

and we conclude that this matrix has rank 2, which is rank deficient.

Let’s review some other features of linear maps that we have encountered in earlier chapters. A square, n × n matrix A of rank n is invertible, which means that there is a matrix that undoes A’s action. This is the inverse matrix, denoted by A−1. See Section 12.4 on how to compute the inverse.

If a matrix is invertible, then it does not reduce dimension and its determinant is nonzero. The determinant of a square matrix measures the volume of the n-dimensional parallelepiped, which is defined by its column vectors. The determinant of a matrix is computed by subjecting its column vectors to a sequence of shears until it is of upper triangular form (forward elimination). The value is then the product of the diagonal elements. (If pivoting takes place, the row exchanges need to be documented in order to determine the correct sign of the determinant.)

14.3 Inner Products

Let u, v, w be elements of a linear space $L_n$ and let a and β be scalars. A map from $L_n$ to the reals ℝ is called an inner product if it assigns a real number ‹v, w› to the pair v, w such that:

$\begin{array}{rrr}\hfill \langle v,w\rangle =\langle w,v\rangle & \hfill \left(\text{symmetry}\right)& \hfill \left(14.3\right)\end{array}$

$\begin{array}{rrr}\hfill \langle \alpha v,w\rangle =\alpha \langle w,v\rangle & \hfill \left(\text{homogeneity}\right)& \hfill \left(14.4\right)\end{array}$

$\begin{array}{lll}\begin{array}{c}\begin{array}{l}\langle u+v,w\rangle =\langle u,w\rangle +\langle v,w\rangle \hfill \end{array}\end{array}\hfill & \left(\text{additivity}\right)\hfill & \left(14.5\right)\hfill \end{array}$

$\begin{array}{lll}\begin{array}{ll}\text{for}\text{all}v\hfill & \langle v,v\rangle \ge 0\text{and}\hfill \end{array}\hfill & & \\ \langle v,v\rangle =0\text{if}\text{and}\text{only}\text{if}v=0\hfill & \left(\text{positivity}\right)\hfill & \left(14.6\right)\hfill \end{array}$

The homogeneity and additivity properties can be nicely combined into

$\langle \alpha u+\beta v,w\rangle =\alpha \langle u,w\rangle +\beta \langle v,w\rangle .$

A linear space with an inner product is called an inner product space.

We can consider the inner product to be a generalization of the dot product. For the dot product in ℝn, we write

$\langle v,w\rangle =v\cdot w=v_1{w}_1+v_2{w}_2+...v_n{w}_n.$

From our experience with 2D and 3D, we can easily show that the dot product satisfies the inner product properties.

Example 14.9

Suppose u, v, w live in ℝ2. Let’s define the following “test” inner product in ℝ2,

$\begin{array}{ll}\langle v,w\rangle =4v_1{w}_1+2v_2{w}_2,\hfill & \left(14.7\right)\hfill \end{array}$

which is an odd variation of the standard dot product. To get a feel for it, consider the three unit vectors, e1, e2, and $r={\left[\begin{array}{ll}1/\sqrt{2}\hfill & 1/\sqrt{2}\hfill \end{array}\right]}^{\text{T}}$, then

$\langle e_1,e_2\rangle =4\left(1\right)\left(0\right)+2\left(0\right)\left(1\right)=0,$

which is the same result as the dot product, but

$\langle e_1,r\rangle =4\left(1\right)\left(\frac{1}{\sqrt{2}}\right)+2\left(0\right)\left(\frac{1}{\sqrt{2}}\right)=\frac{4}{\sqrt{2}}$

differs from $e_1\cdot r=1/\sqrt{2}$. In Figure 14.3 (left), this test inner product and the dot product are graphed for r that is rotated in the range [0, 2 π]. Notice that the graph of the dot product is the graph of cos(θ).

We will now show that this definition satisfies the inner product properties.

$\begin{array}{lll}\hfill \text{Symmetry}:& \hfill \langle v,w\rangle & =4v_1{w}_1+4v_2{w}_2\\ & & =4v_1{w}_1+2v_2{w}_2\\ & & =\langle w,v\rangle .\\ \hfill \text{Homogeneity}:& \hfill \langle \alpha v,w\rangle & =4\left(\alpha v_1\right)w_1+2\left(\alpha v_2\right)w_2\\ & & =\alpha \left(4v_1{w}_1+2v_2{w}_2\right)\\ & & =\alpha \langle v,w\rangle .\\ \hfill \text{Additivity}:& \hfill \langle u+v,w\rangle & =4\left(u_1+v_1\right)w_1+2\left(u_2+v_2\right)w_2\\ & & =\left(4u_1{w}_1+2u_2{w}_2\right)+\left(4v_1{w}_1+2v_2{w}_2\right)\\ & & =\langle u,w\rangle +\langle v,w\rangle .\\ \hfill \text{Positivity}:& \hfill \langle v,v\rangle & =4v_1^2+2v_2^2\ge 0,\\ & \hfill \langle v,v\rangle & =0\text{if}\text{and}\text{only}\text{if}v=0.\end{array}$

We can question the usefulness of this inner product, but it does satisfy the necessary properties.

Inner product spaces offer the concept of length,

$||v||=\sqrt{\langle v,v\rangle }.$

This is also called the 2-norm or Euclidean norm and is denoted as ∥v∥2. Since the 2-norm is the most commonly used norm, the subscript is typically omitted. (More vector norms were introduced in Section 13.2.) Then the distance between two vectors,

$\text{dist}\left(u,v\right)=\sqrt{\langle u-v,u-v\rangle }=||u-v||.$

For vectors in ℝn and the dot product, we have the Euclidean norm

$||v||=\sqrt{v_1^2+v_2^2+...+v_n^2}$

and

$\text{dist}\left(u,v\right)=\sqrt{{\left(u_1-v_1\right)}^2+{\left(u_2-v_2\right)}^2+...+{\left(u_n-v_n\right)}^2}.$

Example 14.10

Let’s get a feel for the norm and distance concept for the test inner product in (14.7). We have

$\begin{array}{c}||e_1||=\sqrt{\langle e_1,e_2\rangle }=4{\left(1\right)}^2+2{\left(0\right)}^2=4\\ \text{dist}\left(e_1,e_2\right)=\sqrt{4{\left(1-0\right)}^2+2{\left(0-1\right)}^2}=\sqrt{6,}\end{array}$

The dot product produces ∥e1∥ = 1 and $\text{dist}\left(e_1,e_2\right)=\sqrt{2}.$

Figure 14.3 illustrates the difference between the dot product and the test inner product. Again, r is a unit vector, rotated in the range [0, 2π]. The middle plot shows that unit length vectors with respect to the dot product are not unit length with respect to the test inner product. The right plot shows that the distance between two vectors differs too.

Orthogonality is important to establish as well. If two vectors v and w in a linear space $L_n$ satisfy

$\langle v,w\rangle =0,$

then they are called orthogonal. If v 1, ..., vn form a basis for $L_n$ and all vi are mutually orthogonal, (vi, vj) = 0 for i ≠ j, then the vi are said to form an orthogonal basis. If in addition they are also of unit length, $||v_i||=1$, they form an orthonormal basis. Any basis of a linear space may be transformed to an orthonormal basis by the Gram-Schmidt process, described in Section 14.4.

The Cauchy-Schwartz inequality was introduced in (2.23) for ℝ2, and we repeat it here in the context of inner product spaces,

${\langle v,w\rangle }^2\le \langle v,v\rangle \langle w,w\rangle .$

Equality holds if and only if v and w are linearly dependent.

If we restate the Cauchy-Schwartz inequality

${\langle v,w\rangle }^2\le {||v||}^2{||w||}^2$

and rearrange,

${\left(\frac{\langle v,w\rangle }{||v||||w||}\right)}^2\le 1,$

we obtain

$-1\le \frac{\langle v,w\rangle }{||v||||w||}\le 1.$

Now we can express the angle θ between v and w by

$\cos \theta =\frac{\langle v,w\rangle }{||v||||w||}.$

The properties defining an inner product (14.3), (14.4), (14.5) and (14.6) suggest

$\begin{array}{c}||v||\ge 0,\\ ||v||=0\text{if}\text{and}\text{only}\text{if}v=0\\ ||\alpha v||=\left|\alpha \right|||v||.\end{array}$

A fourth property is the triangle inequality,

$||v+w||\le ||v||+||w||,$

which we derived from the Cauchy-Schwartz inequality in Section 2.9.

See Sketch 2.22 for an illustration of this with respect to a triangle formed from two vectors in ℝ2. See the exercises for the generalized Pythagorean theorem.

The tools associated with the inner product are key for orthogonal decomposition and best approximation in a linear space. Recall that these concepts were introduced in Section 2.8 and have served as the building blocks for the 3D Gram-Schmidt method for construction of an orthonormal coordinate frame (Section 11.8) and least squares approximation (Section 12.7).

We finish with the general definition of a projection. Let u1, ..., uk span a subspace $L_k$ of $L$. If v is a vector not in that space, then

$Pv=\langle v,u_1\rangle u_1+...+\langle v,u_k\rangle u_k$

is v’s orthogonal projection into $L_k$.

14.4 Gram-Schmidt Orthonormalization

In Section 11.8 we built an orthonormal coordinate frame in ℝ3 using the Gram-Schmidt method. Every inner product space has an orthonormal basis. The key elements of its construction are projections and vector decomposition.

Let b1, ..., br be a set of orthonormal vectors, forming the basis of an r-dimensional subspace ${\mathit{S}}_r$ of $L_n$, where n > r. We want to find br+1 orthogonal to the given bi. Let u be an arbitrary vector in $L_n$, but not in ${\mathit{S}}_r$. Define a vector $\widehat{u}$ by

$\widehat{u}={\text{proj}}_{{\mathit{S}}_r}u=\langle u,b_1\rangle b_1+...+\langle u,b\rangle b_r.$

This vector is u’s orthogonal projection into ${\mathit{S}}_r$. To see this, we check that the difference vector $u-\widehat{u}$ is orthogonal to each of the bi. We first check that it is orthogonal to b1 and observe

$\langle u-\widehat{u},b_1\rangle =\langle u,b_1\rangle -\langle u,b_1\rangle \langle b_1,b_1\rangle -...-\langle u,b_r\rangle \langle b_1,b_r\rangle .$

All terms $\langle b_1,b_2\rangle ,\langle b_1,b_3\rangle ,$ etc. vanish since the bi are mutually orthogonal and $\langle b_1,b_1\rangle =1$. Thus, $\langle u-\widehat{u},b_1\rangle =0$. In the same manner, we show that $u-\widehat{u}$ is orthogonal to the remaining bi. Thus

$b_{r+1}=\frac{u-{\text{proj}}_{{\mathit{S}}_r}u}{||\cdot ||},$

Thus and the set b1, ..., br+1 forms an orthonormal basis for the subspace ${\mathit{S}}_{r+1}$ of $L_n$. We may repeat this process until we have found an orthonormal basis for all of $L_n$.

Sketch 14.1 provides an illustration in which ${\mathit{S}}_2$ is depicted as ℝ2. This process is known as Gram-Schmidt orthonormalization. Given any basis v1, ..., vn of $L_n$, we can find an orthonormal basis by setting $b_1=v_1/||\cdot ||$ and continuing to construct, one by one, vectors b2, ..., bn using the above procedure. For instance,

$\begin{array}{l}{b}_2=\frac{v_2-{\text{proj}}_{{\mathit{S}}_1}{v}_2}{||\cdot ||}=\frac{v_2-\langle v_2,b_1\rangle b_1}{||\cdot ||},\\ b_3=\frac{v_3-{\text{proj}}_{{\mathit{S}}_2}{v}_3}{||\cdot ||}=\frac{v_3-\langle v_3,b_1\rangle b_1-\langle v_3,b_2\rangle b_2}{||\cdot ||}.\end{array}$

Consult Section 11.8 for an example in ℝ3.

Example 14.11

Suppose we are given the following vectors in ℝ4,

$\begin{array}{cccc}{v}_1=\left[\begin{array}{c}1\\ 0\\ 0\\ 0\end{array}\right],& v_2=\left[\begin{array}{c}1\\ 1\\ 1\\ 1\end{array}\right],& v_3=\left[\begin{array}{c}1\\ 1\\ 0\\ 0\end{array}\right],& v_4=\left[\begin{array}{c}0\\ 0\\ 1\\ 0\end{array}\right],\end{array}$

and we wish to form an orthonormal basis, b1, b2, b3, b4.

The Gram-Schmidt method, as defined in the displayed equations above, produces

$\begin{array}{ccc}{b}_1=\left[\begin{array}{c}1\\ 0\\ 0\\ 0\end{array}\right],& b_2=\left[\begin{array}{c}0\\ 1/\sqrt{3}\\ 1/\sqrt{3}\\ 1/\sqrt{3}\end{array}\right],& b_3=\left[\begin{array}{c}0\\ 2/\sqrt{6}\\ -1/\sqrt{6}\\ -1/\sqrt{6}\end{array}\right].\end{array}$

The final vector, b4, is defined as

$\begin{array}{cc}{b}_4=v_4-\langle v_4,b_1\rangle b_1-\langle v_4,b_2\rangle b_2-\langle v_4,b_3\rangle b_3=& \left[\begin{array}{c}0\\ 0\\ 1/\sqrt{2}\\ -1/\sqrt{2}\end{array}\right]\end{array}$

Knowing that the bi are normalized and checking that bi · bj = 0, we can be confident that this is an orthonormal basis. Another tool we have is the determinant, which will be one,

$\left|b_1{b}_2{b}_3{b}_4\right|=1.$

14.5 A Gallery of Spaces

In this section, we highlight some special linear spaces—but there are many more!

For a first example, consider all polynomials of a fixed degree n. These are functions of the form

$p\left(t\right)=a_0+a_{1}t+a_2{t}^2+...+a_n{t}^n$

where t is the independent variable of p(t). Thus we can construct a linear space Pn whose elements are all polynomials of a fixed degree. Addition in this space is addition of polynomials, i.e., coefficient by coefficient; multiplication is multiplication of a polynomial by a real number. It is easy to check that these polynomials have the linearity property (14.1). For example, if p(t) = 3 − 2t + 3t2 and q(t) = −1 + t + 2t2, then 2p(t) + 3q(t) = 3 − t + 12t2 is yet another polynomial of the same degree.

This example also serves to introduce a not-so-obvious linear map: the operation of forming derivatives! The derivative p′ of a degree n polynomial p is a polynomial of degree n − 1, given by

$p^{\prime }\left(t\right)=a_1+2a_{2}t+...+n{a}_n{t}^{n-1}.$

The linear map of forming derivatives thus maps the space of all degree n polynomials into that of all degree n − 1 polynomials. The rank of this map is thus n − 1.

Example 14.12

Let us consider the two cubic polynomials

$\begin{array}{ccc}p\left(t\right)=3-t+2t^2+3t^3& \text{and}& q\left(t\right)=1+t-t^3\end{array}$

in the linear space of cubic polynomials, P3. Let

$r\left(t\right)=2p\left(t\right)-q\left(t\right)=5-3t+4t^2+7t^{3.}$

Now

$\begin{array}{l}{r}^{\prime }\left(t\right)=-3+8t+21t^2,\\ p^{\prime }\left(t\right)=-1+4t+9t^2,\\ q^{\prime }\left(t\right)=1-3t^2.\end{array}$

It is now trivial to check that $r^{\prime }\left(t\right)=2p^{\prime }\left(t\right)-q^{\prime }\left(t\right),$ thus asserting the linearity of the derivative map.

For a second example, a linear space is given by the set of all realvalued continuous functions over the interval [0, 1]. This space is typically named C[0, 1]. Clearly the linearity condition is met: if f and g are elements of C[0, 1], then αf + βg is also in C[0, 1]. Here we have an example of a linear space that is infinite-dimensional, meaning that no finite set of functions forms a basis for C[0, 1].

For a third example, consider the set of all 3 × 3 matrices. They form a linear space; this space consists of matrices. In this space, linear combinations are formed using standard matrix addition and multiplication with a scalar as summarized in Section 9.11.

And, finally, a more abstract example. The set of all linear maps from a linear space $L_n$ into the reals forms a linear space itself and it is called the dual space $L^{*}{}_n$ of $L_n$. As indicated by the notation, its dimension equals that of $L_n$. The linear maps in $L^{*}{}_n$ are known as linear functionals.

For an example, let a fixed vector v and a variable vector u be in $L_n$. The linear functionals defined by ΦV (u) = ‹u, v› are in $L_n^{*}$. Then, for any basis b1, ..., bn of $L_n$, we can define linear functionals

$\begin{array}{cc}{\Phi }_{b_i}\left(u\right)=\langle u,b_i\rangle & \text{for}i=1,...,n.\end{array}$

These functionals form a basis for $L^{*}{}_n$.

Example 14.13

In ℝ2, consider the fixed vector

$v=\left[\begin{array}{c}1\\ -2\end{array}\right].$

Then ΦV (u) = (u, v) = u1 − 2u2 for all vectors u, where $\langle \cdot ,\cdot \rangle$ is the dot product

Example 14.14

Pick e1, e2 for a basis in ℝ2. The associated linear functionals are

$\begin{array}{cc}{\Phi }_{e_1}\left(u\right)=u_1,& {\Phi }_{e_2}\left(u\right)=u_{2.}\end{array}$

Any linear functional Φ can now be defined as

$\Phi \left(u\right)=r_1{\Phi }_{e_1}\left(u\right)+r_2{\Phi }_{e_2}\left(u\right),$

where r1 and r2 are scalars.

14.6 Exercises

1. Given elements of ℝ4,

   $\begin{array}{ccc}u=\left[\begin{array}{c}1\\ 2\\ 0\\ 4\end{array}\right],& v=\left[\begin{array}{c}0\\ 0\\ 2\\ 7\end{array}\right],& w=\left[\begin{array}{c}3\\ 1\\ 2\\ 0\end{array}\right],\end{array}$

   is r = 3u + 6v + 2w also in ℝ4?

2. Given matrices that are elements of $M_{3\times 3}$,

   $\begin{array}{ccc}A=\left[\begin{array}{ccc}1& 0& 2\\ 2& 0& 1\\ 1& 1& 3\end{array}\right]& \text{and}& B=\left[\begin{array}{ccc}3& 0& 0\\ 0& 3& 1\\ 4& 1& 7\end{array}\right],\end{array}$

   is C = 4A + B an element of $M_{3\times 3}$?

3. Does the set of all polynomials with an = 1 form a linear space?
4. Does the set of all 3D vectors with nonnegative components form a subspace of ℝ3?

5. Are

   $\begin{array}{ccc}u=\left[\begin{array}{c}1\\ 0\\ 1\\ 0\\ 1\end{array}\right],& v=\left[\begin{array}{c}1\\ 1\\ 1\\ 1\\ 1\end{array}\right],& w=\left[\begin{array}{c}0\\ 1\\ 0\\ 1\\ 0\end{array}\right]\end{array}$

   linearly independent?

6. Are

   $\begin{array}{ccc}u=\left[\begin{array}{c}1\\ 2\\ 1\end{array}\right],& v=\left[\begin{array}{c}3\\ 6\\ 1\end{array}\right],& w=\left[\begin{array}{c}2\\ 2\\ 1\end{array}\right]\end{array}$

   linearly independent?

7. Is the vector

   $r=\left[\begin{array}{c}2\\ 3\\ 2\\ 3\\ 2\end{array}\right]$

   in the subspace defined by u, v, w defined in Exercise 5?

8. Is the vector

   $r=\left[\begin{array}{c}2\\ 3\\ 2\\ 3\\ 2\end{array}\right]$

   in the subspace defined by u, v, w defined in Exercise 6?

9. What is the dimension of the linear space formed by all n × n matrices?

10. Suppose we are given a linear map A : ℝ4 → ℝ2, preimage vectors vi, and corresponding image vectors wi. What are the dimensions of the matrix A ? The following linear relationship exists among the preimages,

    $v_4=3v_1+6v_2+9v_3.$

    What relationship holds for w4 with respect to the wi?

11. What is the rank of the matrix

    $\left[\begin{array}{rrr}\hfill 1& \hfill 2& \hfill 0\\ \hfill -1& \hfill -2& \hfill 1\\ \hfill 0& \hfill 0& \hfill 1\\ \hfill 2& \hfill 4& \hfill -1\end{array}\right]?$

12. What is the rank of the matrix

    $\left[\begin{array}{rrrrr}\hfill 1& \hfill 2& \hfill 0& \hfill 0& \hfill 0\\ \hfill -1& \hfill 0& \hfill 0& \hfill -2& \hfill 1\\ \hfill 0& \hfill 0& \hfill 1& \hfill 0& \hfill 1\end{array}\right]?$

13. Given the vectors

    $\begin{array}{rrr}\hfill w=\left[\begin{array}{r}\hfill 1/\sqrt{10}\\ \hfill 3/\sqrt{10}\end{array}\right]& \hfill \text{and}& \hfill r=\left[\begin{array}{r}\hfill -3/\sqrt{10}\\ \hfill 1/\sqrt{10}\end{array}\right],\end{array}$

    find $\langle w,r\rangle ,||w||,||r||$, and dist(w, r) with respect to the dot product and then for the test inner product in (14.7).

14. For v, w in ℝ3, does

    $\langle v,w\rangle =v_1^2{w}_1^2+v_2^2{w}_2^2+v_3^2{w}_3^2$

    satisfy the requirements of an inner product?

15. For v, w in ℝ3, does

    $\langle v,w\rangle =4v_1{w}_1+v_2{w}_2+2v_3{w}_3$

    satisfy the requirements of an inner product?

16. In the space of all 3 × 3 matrices, is

    $\langle A,B\rangle =a_{1,1}{b}_{1,1}+a_{1,2}{b}_{1,2}+a_{1,3}{b}_{1,3}+a_{2,1}{b}_{2,1}+...+a_{3,3}{b}_{3,3}$

    an inner product?

17. Let p(t) = po + p1t + p2t2 and q(t) = q0 + q1t + q2t2 be two quadratic polynomials. Define

    $\langle p,q\rangle =p_0{q}_0+p_1{q}_1+p_2{q}_2.$

    Is this an inner product for the space of all quadratic polynomials?

18. Show that the Pythagorean theorem

    ${||v+w||}^2={||v||}^2={||w||}^2$

    holds for orthogonal vectors v and w in an inner product space.

19. Given the following vectors in ℝ3,

    $\begin{array}{rrr}\hfill v_1=\left[\begin{array}{r}\hfill 1\\ \hfill 1\\ \hfill 0\end{array}\right],& \hfill v_2=\left[\begin{array}{r}\hfill 0\\ \hfill 1\\ \hfill 0\end{array}\right],& \hfill v_3=\left[\begin{array}{r}\hfill -1\\ \hfill -1\\ \hfill -1\end{array}\right],\end{array}$

    form an orthonormal basis, b1, b2, b3.

20. Given the following vectors in ℝ4,

    $\begin{array}{rrrr}\hfill v_1=\left[\begin{array}{r}\hfill 0\\ \hfill 0\\ \hfill 1\\ \hfill 0\end{array}\right],& \hfill v_2=\left[\begin{array}{r}\hfill 1\\ \hfill 1\\ \hfill 1\\ \hfill 1\end{array}\right],& \hfill v_3=\left[\begin{array}{r}\hfill 1\\ \hfill 1\\ \hfill 0\\ \hfill 0\end{array}\right],& \hfill v_4=\left[\begin{array}{r}\hfill 1\\ \hfill 0\\ \hfill 0\\ \hfill 0\end{array}\right],\end{array}$

    form an orthonormal basis, b1 = v1, b2, b3, b4.

21. Find a basis for the linear space formed by all 2 × 2 matrices.
22. Does the set of all monotonically increasing functions over [0, 1] form a linear space?
23. Let $L^{*}$ be a linear space. Is the map Φ(u) = ∥u∥ an element of the dual space $L^{*}$?
24. Show the linearity of the derivative map on the linear space of quadratic polynomials P2.

1Note that we will not always use the $L$ notation, but rather the standard name for the space when one exists.