Vector Spaces

There are many vector spaces other than the spaces that we have been using heretofore. What type of mathematical constructs will be called “vector spaces”? The structure they must have is set out in the following formal definition.

The reader will notice that these requirements are exactly the salient properties of the spaces that we called attention to in Section 2.1.

We have already noticed that, with the definitions adopted in Section 2.1, is a vector space. Thus, we have at once an infinite number of different vector spaces, because n can be any natural number.

The word scalar occurs frequently in the context of vector spaces. It almost always will mean a real number, but in some situations there is good reason for allowing complex numbers as scalars. (A complex number is of the form α + iβ, where α and β are real numbers and i² = −1. See Appendix B.)

It turns out that the element in Axiom 5 is unique for each vector u. It is usually denoted by −u. The axiom then states that (−u) + u = 0 for all u. We further drop the parentheses and use Axiom 2 in this last equation to arrive at u − u = 0.

Theorems on Vector Spaces

To illustrate the deductions that can be drawn from the preceding axioms, we consider five theorems valid in all vector spaces. We can use only the axioms to prove Theorem 1. But then we can use the axioms and Theorem 1 to prove Theorem 2, and so on.

PROOF Let x = c0. We want to prove that x = 0.

See General Exercise 19 for another proof of Theorem 1.

PROOF Assume the hypotheses, and suppose that c ≠ 0. Then c⁻¹ exists, and we can multiply the equation cx = 0 by c⁻¹, arriving at 1 · x = c⁻¹0. By Theorem 1 and Axiom 10, this becomes x = 0.

PROOF The crucial property of is that + x = 0. Suppose, then, that for some x we have both u + x = 0 and v + x = 0. Can we conclude that u must equal v? It is probably not obvious, but one proof goes like this:

Notice that in this proof each equality requires specific justification and the axioms are used in the order indicated.

PROOF One proof proceeds as follows, and the axioms used are indicated in each step.

Verify the use of the axioms in the order indicated.

PROOF Following the proof above, we have

Consequently = (−1)x by Theorem 3.

Before going on, some remarks should be made. First, the element is written as −x or (−1)x. Theorem 5 justifies this. Second, the structure that is described by the ten axioms is technically a real vector space. This means that the constants appearing in Axioms 6–10 are real numbers. The real numbers in this context are often called scalars to distinguish them from the elements of V, which are then called vectors. (Thus, the answer to the question “What is a vector?” can be given truthfully as “It is an element of avectorspace.”)

Later in the book, we shall broaden our perspective and discuss vector spaces with complex numbers as the scalars. Certain problems in vector space theory (notably eigenvalue problems) may have no solution if we restrict ourselves to the real numbers as scalars. When one speaks of a real vector space, it means only that the scalars are taken to be real numbers. In principle, other fields can be used for the scalars. If a particular field, F, is used, one speaks of a vector space over the field F. For information about fields, consult a textbook on abstract algebra or search the World Wide Web on the Internet.

Various Examples

We present a number of examples to illustrate these concepts.

SOLUTION Some elementary polynomials are p₀(t) = 1, p₁(t) = t, p₂(t) = t², and p₃(t) = t³. Notice that any polynomial of degree at most 3 is a linear combination of the four polynomials p₀, p₁, p₂, and p₃. Thus, we can write

The set we have described is a vector space—it being assumed that the usual algebraic operations are used. One can add two polynomials of degree at most 3, and the result is another polynomial of degree at most 3. The zero element in this vector space is the polynomial defined by p(t) = 0 for all t. Notice that the notation p₂ is the name of the function whose value at t is given by p₂(t) = t². One should not refer to t² as a polynomial. It is nothing but a real number, because t is understood to be a real number. The notation t t² can also be used to specify the function p₂. We do not stop to verify all the vector-space axioms for this vector space. (They are very easy.)

In the same way as in Example 1, for each nonnegative integer n, we have a space consisting of all polynomials of degree not exceeding n. This space is usually written . Now we have two infinite lists of distinct vector spaces, and , where n = 1, 2, 3, ….

EXAMPLE 2

Consider the set of all continuous real-valued functions defined on the interval [−1, 1]. Is this a vector space if we adopt the standard definitions for the algebraic operations?

SOLUTION Why is the first axiom true in this space? If we add two continuous functions, the result is another continuous function, by a theorem in calculus. A similar remark concerns the multiplication of a continuous function by a scalar. Thus, the two closure axioms are true by some theorems in calculus. The remaining axioms are all easily verified.

The space in Example 2 is customarily given the designation C[−1, 1]. More properly, one should write C([−1, 1]), because for an arbitrary domain, D, we would write C(D). As in Example 1, one should make a careful distinction between a function f and one of its values, f(t). In this book, we try to adhere to these standards, and the reader is encouraged to do the same.

EXAMPLE 3

Consider the set of all infinite sequences

If we use the natural definitions of vector addition and scalar multiplication, is this a vector space?

SOLUTION Yes. The closure axioms are true, just as they are for . In fact, we have here an obvious extension of those familiar spaces. This new space should be named . Notice that a sequence x is really a function. Its domain is the set of natural numbers, = {1, 2, 3,…}. We could use either the notation x(k) or x_k or x_k for a generic component of the vector x. In set theory, this space would also be denoted by . (In general, the set of all functions from one set B into another set A is denoted by A^B. For example, is the set of all maps from the natural numbers to the set S.) The vectors in can be interpreted as signals in electrical engineering.

EXAMPLE 4

Consider the set of all continuous real-valued functions f defined on the entire real line, (−∞, +∞), and having the property

Is this a vector space?

SOLUTION It is understood that the natural definitions of the algebraic operations are used. Let us verify the first axiom for this vector space. Suppose that f and g are two elements of this space. Is their sum also in this space? The answer is yes by the triangle inequality:

It sometimes happens that one vector space is a subset of a larger vector space and shares the definitions of vector addition, scalar multiplication, 0, and so on. For example, the set, X, of all vectors in that have the form (x₁, x₂, 0) is a vector space, as is easily verified, and it is obviously a subset of . It has the same vector addition and scalar multiplication that has. We say that X is a subspace of . (Being a subspace is more than being a subset. Do you see why?) The next example illustrates this same phenomenon, and the topic arises again in Section 5.1.

SOLUTION Recall from Section 1.3 that

The kernel of A is obviously a subset of . But is it a vector subspace? The only axioms that require verification are the two closure Axioms 1 and 6, and Axiom 4 concerning the zero element. All of the other axioms are automatically fulfilled because we know that is a vector space. To establish Axioms 1 and 6 together, simply write

Thus, if x and y are in the kernel of A, then so is αx + βy. Axiom 4 is obviously true, since A0 = 0. We say that the kernel of A is a subspace of .

Linearly Dependent Sets

The concepts of linear dependence and linear independence play a crucial role in linear algebra, as noted in Section 1.3. These terms apply to sets of vectors, not to single vectors, and that fact by itself makes for difficulties in comprehension.

(The cited equation has only a finite number of terms.) The term nontrivial in this context means that at least one of the coefficients c_i is not zero. That condition can be expressed by writing .

A number of examples follow, to help in assimilating this concept.

SOLUTION It is automatically linearly dependent, because 1 · 0 = 0. This is a nontrivial equation by our definition (at least one coefficient is not zero).

EXAMPLE 7

Consider the set of rows in the matrix

Is it linearly dependent?

SOLUTION Yes. We observe that −3r₁ + 2r₂ − r₃ = 0, where r_i is the ith row in the matrix.

SOLUTION No. We notice that 2p₁ − 3p₂ − p₃ = 0.

EXAMPLE 9

Is the set of these three matrices linearly independent?

SOLUTION The answer is no, because C = 3A − B.

SOLUTION For the first set, we consider the equation c₁u₁ + c₂u₂ + c₃u₃ = 0. The issue is whether this equation has a nontrivial solution. The coefficient matrix for the problem is

Here we do not include the righthand side because the system of equations is homogeneous and the righthand side consists of zeros. Hence, we obtain c₁ = c₂ = c₃ = 0, and the set is linearly independent. For the second set, we look at just the first two columns in the preceding matrix. Of course, the conclusion is the same.

SOLUTION How can we identify these 2 × 2 matrices with column vectors so that we can treat them in the same manner as in Example 10? We make them into vectors, which exactly correspond to each matrix:

Because the only solution is [0, 0, 0]^T, the given set of matrices is linearly independent.

Later, in Section 5.2, we shall justify fully the method used in Example 11. It hinges on the fact that is isomorphic to .

DEFINITION

Consider a finite, indexed set of vectors {u₁, u₂, …, u_m} in a vector space. We say that the indexed set is linearly dependent if there exist scalars c_i such that

If the indexed set is not linearly dependent, we say that it is linearly independent. The expression implies that at least one c_i is nonzero.

EXAMPLE 12

Is the indexed set of rows in the matrix linearly dependent?

SOLUTION Label the rows of A as r₁, r₂, and r₃. This is understood to be an indexed set. (Each vector has an index attached to it.) In the definition of linear dependence, we can take c₁ = 1, c₂ = 0, and c₃ = −1 because r₁ − r₃ = 0. The indexed set of rows contains three rows with the vector (2, 5, 7) appearing twice. If we are interested only in the set, we need not mention this vector twice. Thus, the set consists of two vectors, (2, 5, 7) and (4, 1, −5). There is a difference between the set and the indexed set. In this example, the latter is linearly dependent, whereas the former is linearly independent. In an indexed set, we consider the vector u₁ =(2, 5, 7) to be different from u₃ = (2, 5, 7) because their indices (1 and 3) are different.

In an effort to avoid confusion we always consider the rows (or the columns) of a matrix to be indexed sets.

EXAMPLE 13

Is the set of columns in the following matrix linearly dependent?

SOLUTION It is linearly independent, because when we try to solve the equation Ax = 0 for x, we discover that x = 0 is the only solution.

SOLUTION Apply the technique of Example 13 to the transposed matrix

The equation A^Tx = 0 has only the trivial solution (x = 0), and the set in question is linearly independent. Later we will prove theorems that make such questions easier to answer.

PROOF Let x be a point in the span of S, where we suppose S to be a linearly independent set. Then , for appropriate a_i ∈ and u_i ∈ S. Suppose there is another such representation, , where b_i ∈ and v_i ∈ S. Put U = {u₁, u₂, …, u_n} and V = {v₁, v₂, …, v_m}. Consider

(Here, k is not necessarily n + m.) All the points w_i are different from each other and belong to S. Because S is linearly independent, so is U ∪ V. By supplying zero coefficients, if necessary, we can write and . By subtraction, we get . Because U ∪ V is linearly independent, all the coefficients in this last equation are zero. Hence, we obtain c_i = d_i for all i.

SOLUTION If we take a linear combination of these columns with coefficients (−3, 5, −2), we get the same vector as when we use coefficients (13, −14, 7). (In both cases the result is the vector [5, 3, 5]^T.) By Theorem 6, the columns in question must form a linearly dependent set.

PROOF Let A be m × n, and n > m. Think of what happens if we try to solve the equation Ax = 0. This is a homogeneous equation with more variables (n) than equations (m). By Corollary 2 in Section 1.3, such a system must have nontrivial solutions.

SOLUTION Put the vectors into a matrix as columns. The resulting matrix will have three rows and four columns:

By Theorem 7, the set of columns is linearly dependent.

PROOF Put the vectors as columns into an n × m matrix, and apply Theorem 7.

PROOF By hypothesis, there is a relation , in which the coefficients c_i are not all 0. Let r be the last index for which c_r ≠ 0. Then c_i = 0 for i > r and . Because c_r ≠ 0, this last equation can be solved for u_r, showing that u_r is a linear combination of the preceding vectors in the set.

PROOF Let {v₁, v₂, …, v_n} span a vector space V, and let {u₁, u₂, …, u_m} be any subset of V such that m > n. By the definition of span, there exist coefficients a_ij such that for 1 ≤ i ≤ m. Consider the homogeneous system of equations . This system has fewer equations than unknowns. Therefore, Corollary 2 in Section 1.3 is applicable, and there must exist a nontrivial solution, x = (x₁, x₂, ···, x_m). Now we see that the set of vectors u_i is linearly dependent, because

PROOF If that conclusion is false, there must exist a nontrivial equation of the form where the points v_i belong to S and the c_i are scalars, not all zero. If c₀ = 0, then that equation will contradict the linear independence of the set S. Hence, we obtain c₀ ≠ 0 and , showing that x is in the span of S.

SOLUTION The answer is yes and deserves a proof. We shall use the method of contradiction to carry out this proof. Suppose therefore that the set of functions p_j is linearly dependent. (Then we hope to arrive at a contradiction.) From the definition of linear dependence, there must exist a nontrivial equation of the form

(In this context, nontrivial means that the coefficients c_j are not all zero.) Think about this displayed equation. Does it not say that a certain nontrivial polynomial of degree at most m is, in fact, equal everywhere to 0? One need only recall from the study of algebra that a nontrivial polynomial of degree k can have at most k zeros (or roots). See Appendix B, Theorem B.1. We have reached a contradiction.

Observe that linear dependence always involves a linear combination of vectors, and a linear combination is always restricted to a finite set of terms. We cannot consider infinite series of vectors without bringing convergence questions into play, and that is another story altogether.

Linear Mapping

The concept of a linear mapping was introduced in Section 2.3. It is meaningful for maps between any two vector spaces. The characteristic property is T(ax + by) = aT(x) + bT(y). Let us illustrate this matter with a linear space of polynomials.

Consider the space , consisting of all polynomials whose degree does not exceed 3. (This is an important example of a vector space.) We can use the following natural basic polynomials for this space: p_j(t) = t^j, where 0 ≤ j ≤ 3. Thus, we have p₀(t) = 1, p₁(t) = t, p₂(t) = t², and p₃(t) = t³. Every element of is a unique linear combination of these four basic polynomials. For example, the polynomial f defined by f(t) = 7 − 3t + 9t² − 5t³ is expressible as f = 7p₀ − 3p₁ + 9p₂ − 5p₃. Consequently, if g is a polynomial of degree at most 3, and if , then we can represent g by the vector (c₀, c₁, c₂, c₃). If T is a linear transformation from into some other vector space, we will have

This easy calculation shows that a linear map is completely determined as soon as its values have been prescribed on a set that spans the domain.

Caution: The values assigned are not necessarily unrestricted.

EXAMPLE 18

For a concrete case of the preceding idea, suppose that T maps into according to the following rules:

What is the matrix for this transformation?

SOLUTION For u = c₀p₀ + c₁p₁ + c₂p₂ + c₃p₃, we have

SOLUTION We note that the representation of this polynomial by a vector in is (−5, 3, −2, 4). (One must write the terms in the correct order.) The answer to the question is therefore

A theorem to formalize what we have seen in the preceding examples is as follows.

PROOF The linear transformation will be defined on the set U = Span{u₁, u₂, … u_n} by this equation:

We must recall that any vector in U has one and only one representation as a linear combination of the vectors u_i.

SOLUTION Let T be such a map. Being linear, T must obey the equation T(cx) = cT(x). Hence, we have

The answer to the question posed is no.

Theorem 12 does not apply in Example 20 because the pair of vectors (1, 0) and (3, 0) is not linearly independent. A slightly different point of view is that (3, 0) − 3(1, 0) = 0 and therefore by linearity we must have T(3, 0) − 3T(1, 0) = 0. This leads to the following theorem.

Consequently, if {u_i} is linearly dependent, then {T(u_i)} is linearly dependent, and if {T(u_i)} is linearly independent, then {u_i} is linearly independent.

Application: Models in Economic Theory

Linear algebra has a large role to play in economic theory. Two simple, yet basic, economic models are discussed here.

Think of a nation’s economy as being divided into n sectors, or industries. For example, we could have the steel industry, the shoe industry, the medical industry, and so on. (In a realistic model, there could be 1000 or more sectors.) An n × n matrix A = (a_ij) is given; it contains data concerning the productivity of the n sectors. Specifically, a_ij is the fraction of the output from industry j that goes to industry i as input in one year. We assume in a closed model that all the output of the various industries remains in the system and is purchased by the sectors in the system. Because of this assumption (that the system is closed), the column sums in the matrix A are 1:

Let x_j be the value placed on the output from the jth sector in one year. The price vector x = (x₁, x₂, …, x_n) is unknown but will be determined after certain requirements have been imposed on it.

An equilibrium state occurs if the price vector x is a nonzero vector x = (x₁, x₂, …, x_n) such that Ax = x. Notice that if x has this property, then any nonzero scalar multiple of x will also have that property.

PROOF The equation in question is (I − A)x = 0. The coefficient matrix here is singular (noninvertible) because its rows add up to 0. Consequently, the kernel of I − A contains nonzero vectors.

This model is called the Leontief Closed Model in honor of Wassily Leontief.⁵ The model described is said to be closed because the sectors in the model satisfy the needs of all sectors, but no goods leave or enter the system.

All of the elements of the matrix A are nonnegative, and the elements in each column sum to 1. The matrix A occurring here is called an input– output matrix or consumption matrix. If x is a solution of the equation, then scalar multiples of x will also be solutions. The equation Ax = x indicates that x is a fixed point of the mapping x Ax. Solving the equation (I − A)x = 0 is easily done: we are looking for a nontrivial solution to a system of homogeneous equations.

To illustrate the Leontief Closed Model, we consider an example involving three sectors S₁, S₂, and S₃. The pertinent information is given in this table:

The entries in the table form the matrix A. Notice that the column sums are all 1. The consumption matrix, I − A, is shown here along with its reduced echelon form:

The general solution of this homogeneous system is x₁ = x₃ and , where x₃ is a free variable. One simple solution is x = (4, 3, 4). (All solutions are multiples of this one.) The rows in the input–output table above show how the three commodities go from source to user (seller to buyer) in the economy. The flow of commodities between the different sectors can be shown in a graph as in Figure 2.27. Row 1 of the table indicates that for sector S₁ to produce one unit of its product it must consume one-fourth of its own product, one-third of sector S₂’s product, and one-half of sector S₃’s product. There are similar flows in the other sectors.

Usually, an economy has to satisfy some outside demands from non-producing sectors such as government agencies. In this case, there is also a final demand vector b = (b_i), which lists the demands placed upon the various industries for output unconnected with the industries mentioned previously. With x_i and c_ij as previously defined, we are led to the equations: a_i1x₁ + a_i2x₂ + ··· + a_inx_n + b_i = x_i. The linear system for the Leontief Open Model is

FIGURE 2.27 Flow among three sectors.

The term Ax is called the intermediate demand vector and b is the final demand vector. In this case, the elements of the matrix A are nonnegative and the column sums are less than 1.

To illustrate the Leontief Open Model, we consider a simple example involving two industrial sectors S₁ and S₂ as given by this table:

Suppose that the initial demand vector is b = (350, 1700). Then we solve the system

and obtain the solution x = (1000, 2000). The flow between the different sectors in this input–output table can be illustrated by the graph shown in Figure 2.28.

FIGURE 2.28 Flow example between two sectors.

A common question in input–output analysis is: Given a forecasted demand, how much output from each of the sectors would be needed to satisfy the final demand? If we increase the final demand to $400 for sector S₁ and decrease it to $1600 for sector S₂, we have the new demand vector b = (400, 1600). Here the change in final demand is Δb = (50, −100). Solving the system

we find Δx = (29.70, −99.01), which is the change in the output vector. This change in demand requires an increase in production by sector S₁ and a decrease in S₂. Consequently, the new output vector is x ≈ (1030, 1900). This process can be repeated for different possible situations.

In Section 8.3, the Leontief models are studied in greater detail.

SUMMARY 2.4

KEY CONCEPTS 2.4

Vector spaces, axioms for vector spaces, spaces of sequences, spaces of functions, spaces of polynomials, linear independence, linear dependence, spaces of matrices, vector subspaces, indexed set, Leontief Closed Model

GENERAL EXERCISES 2.4

COMPUTER EXERCISES 2.4

1. Use mathematical software to solve this Leontief system involving

2. Suppose that an economy consists of four sectors, labeled S_a, S_b, S_c, and S_d. The interactions between the sectors are governed by a table as follows: