4.1 Vector Spaces and Subspaces
A non-empty set \(V\) is called a vector space if there are defined on \(V\) two operations, addition of vectors and multiplication by scalars, so that ten basic properties hold for all vectors in the space. These properties are listed here, (you are not required to memorize them).
Let \(u\), \(v\), and \(w\) be vectors in \(V\), and let \(c\) and \(d\) be scalars (real numbers).
- \(u+v\) is in \(V\).
- \(cu\) is in \(V\).
- \(u+v=v+u\).
- \((u+v)+w=u+(v+w)\)
- \(0+u=u=u+0\).
- There exists a vector \(-u\) such that \(u+(-u)=-u+u=0\).
- \(c(u+v)=cu+cv\).
- \((c +d)u=cu+du\).
- \(c(du)=(cd)u\).
- \(1u=u\).
That list of properties is long but here is a summary:
- The sum of any two vectors in \(V\) is also in \(V\).
- Any scalar multiple of a vector in \(V\) is also in \(V\). This includes \(0v\).
- The last 8 can be stated as “Addition and scalar multiplication are well behaved.”
Examples of vector spaces:
- \(\mathbb{R}^n\)
- \(\mathbb{P}_n\), the set of polynomials of degree at most \(n\).
A subspace \(H\) of a vector space \(V\) is a subset of \(V\) which is also a vector space. This means that \(H\) must contain the zero vector and must be closed under addition and scalar multiplication.
\(H\) = Span\(\left\{\begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}\right\}\) is a subspace of \(\mathbb{R}^3\)
Check the properties: Given two vectors \(u = a_1\begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} +a_2 \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}\) and \(v = b_1\begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} +b_2 \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}\)
Closed? Is \(u + v\) in \(H\)?
Zero vector?
Some lines are subspaces and some are not.
- The line \(x_2 = a x_1\) is a subspace of \(\mathbb{R}^2\).
- The line \(x_2 = a x_1 + b\) is NOT a subspace of \(\mathbb{R}^2\).
Why?
Is \(\mathbb{R}^2\) a subspace of \(\mathbb{R}^3\)?
The set \(W\) = {the 1st and 3rd quadrants of the plane}. Is \(W\) a subspace of \(\mathbb{R}^2\)?
A basis for a vector space is a set of linearly independent vectors that generate the space.
A basis is a minimal spanning set.
\(\left\{ \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \begin{bmatrix} 2 \\ 1 \end{bmatrix}, \begin{bmatrix} 1 \\ 2 \end{bmatrix} \right\}\) is NOT a basis for \(\mathbb{R}^2\)
\(\left\{ \begin{bmatrix} 1 \\ 1 \end{bmatrix}, \begin{bmatrix} 2 \\ 1 \end{bmatrix} \right\}\) is a basis for \(\mathbb{R}^2\)
Is the set of vectors of the form \(\begin{bmatrix} a-b \\ b-c \\ c-a \\b \end{bmatrix}\) a vector space? If it is, find a basis.
Is the set of vectors of the form \(\begin{bmatrix} 3a+b \\ 4 \\ a-5b \end{bmatrix}\) a vector space? If it is, find a basis.
4.2 Null Spaces, Row Spaces and Column Spaces
Recall:
A subspace \(H\) of a vector space \(V\) is a set of vectors in \(V\) which is closed under addition and scalar multiplication.
Is the set of solutions to the matrix equation \(Ax=0\) for \(x \in \mathbb{R}^n\) a vector space?
Solution: zero vector? Yes, \(x=0\) is a solution to the equation.
Closed? Suppose \(u\) and \(v\) are solutions to \(Ax=0\). What about \(u+v\)?
\[\begin{align*}
A(u+v) & = Au + Av\\
&= 0 + 0
\end{align*}\]
Therefore closed.
The set of solutions to the equation \(Ax=0\) is a subspace of \(\mathbb{R}^n\) called the null space of matrix \(A\).
The null space of a matrix \(A\) (Nul \(A\)) is the span of the vectors obtained by solving \(Ax=0\).
\[\text{Nul } A = \{x: x \text{ is in }\mathbb{R}^n \text{ and } Ax = 0\}\]
Suppose the reduced echelon form for a matrix \(A\) is
\[\begin{bmatrix}
1 & 3 & 0 & -1 & 8\\
0 & 0 & 1 & -2 & 1 \\
0 & 0 & 0 & 0 & 0
\end{bmatrix}\]
Find the solution set of the equation \(Ax = 0\). (i.e. Find Nul \(A\))
Consider the linear transformation \(T(x) = Ax\) where
\[A =\begin{bmatrix}
1 & 2 & -1\\
2 & 5 & 0
\end{bmatrix}\]
- Is the vector \(x = \begin{bmatrix}1 \\ 3 \\ 1\end{bmatrix}\) in Nul \(A\)?
- Find a description of Nul \(A\).
The column space of a matrix \(A = [a_1 ~ a_2~ \cdots ~ a_n]\) (Col \(A\)) is the span of the columns of \(A\).
\[\text{Col } A = \text{Span } \{a_1, a_2, \ldots, a_n\}\]
\[\text{Col } A = \{b: b = Ax \text{ for some } x \in \mathbb{R}^n \}\]
(Thm 6 P. 226) The pivot columns of matrix \(A\) span Col \(A\) and is the smallest set of vectors that will span Col \(A\).
The row space of a matrix \(A\), which is denoted Row \(A\), is the set of all linear combinations of the rows of \(A\).
Another way of saying this is: The row space of \(A\) is the span of the rows of \(A\).
(Thm 7 P. 227) If two matrices \(A\) and \(B\) are row equivalent, or symbolically \(A \sim B\), then their row spaces are the same. If \(B\) is in echelon form, the nonzero rows of \(B\) form a basis for the row space of \(A\) as well as that for \(B\).
Consider the matrix
\[A =\begin{bmatrix}
1 & 2 & -1\\
2 & 5 & 0
\end{bmatrix}\]
- What is Col \(A\)?
- What is the smallest representation of Col \(A\)?
- What is Row \(A\)?
\(A\) is an \(m \times n\) matrix. Mark each statement TRUE or FALSE.
The null space of \(A\) is the solution set of the equation \(Ax=0\).
The null space of an \(m \times n\) matrix is in \(\mathbb{R}^m\)
The column space of \(A\) is the range of the mapping \(x \mapsto Ax\).
Col \(A\) is the set of all vectors that can be written as \(Ax\) for some \(x\).
A null space is a vector space.
The column space of an \(m \times n\) matrix is in \(\mathbb{R}^m\).
Col \(A\) is the set of all solutions of \(Ax=b\).
The range of a linear transformation is a vector space.
Let
\[A = \begin{bmatrix} -5 & -15 & 2 & 1 & -38\\ -3 & -9 & 1 & 1 & -23\\ -2 & -6 & 2 & -2 & -14 \end{bmatrix}\]
Find Nul \(A\), Col \(A\), Row \(A\) and the simplest form of Col \(A\). The reduced echelon form of \(A\) is:
\[\begin{bmatrix}
1 & 3 & 0 & -1 & 8\\
0 & 0 & 1 & -2 & 1 \\
0 & 0 & 0 & 0 & 0
\end{bmatrix}\]
Find the row space, column space and null space of
\[A =\begin{bmatrix} 1 & 2 & 0 & -1 \\ 0 & -1 & 1 & 1 \\ 1 & 1 & 1 & 0 \end{bmatrix}\]
4.3 Basis and linearly independent sets
Recall:
An indexed set of vectors \(\{v_1, v_2, \ldots, v_n\}\) is said to be linearly independent if the equation \[a_1 v_1+ a_2 v_2+ \cdots +a_n v_n=0\] has only the trivial solution \(a_1 = a_2 = \cdots = a_n = 0\). It is said to be linearly dependent otherwise.
Let \(H\) be a vector space (including possibly that \(H\) is a subspace of some other vector space). A set of vectors \[\mathcal{B} = \{b_1, b_2, \ldots, b_n\}\] is a basis for \(H\) if:
- \(\mathcal{B}\) is a linearly independent set of vectors. (This means there is a pivot position in every column of the reduced matrix consisting of the columns of \(\mathcal{B}\)).
- \(H =\) Span \(\mathcal{B}\).
Let \(v_1 = \begin{bmatrix} 3\\0\\-6 \end{bmatrix}\), \(v_2 = \begin{bmatrix} -4\\1\\7 \end{bmatrix}\) and \(v_3 = \begin{bmatrix} -2\\1\\5 \end{bmatrix}\). Do \(\{v_1, v_2, v_3\}\) form a basis for \(\mathbb{R}^3\)?
Spanning Set Theorem (Chapter 4, Theorem 5)
Let \(S = \{v_1, v_2, \ldots, v_p\}\) be a set of vectors in \(\mathbb{R}^n\) and let \(H =\) Span \(\{v_1, v_2, \ldots, v_p\}\).
- If one of the vectors \(v_k\) is a linear combination of the remaining vectors, then the set formed from \(S\) by removing \(v_k\) still spans \(H\).
- If \(H\neq\{0\}\), then some subset of \(S\) is a basis for \(H\).
Find a basis for Nul \(B\), Row \(B\) and Col \(B\) where
\[B =\begin{bmatrix} 1 & 4 & 0 & 2 & -1 \\ 3 & 12 & 1 & 5 & 5 \\ 2 & 8 & 1 & 3 & 2 \\ 5 & 20 & 2 & 8 & 8 \end{bmatrix}\]
4.4 Coordinate Systems
If a vector space \(V\) has a basis \(\mathcal{B}=\{b_1, \ldots, b_n\}\), then any set in \(V\) containing more than \(n\) vectors must be linearly dependent.
Any vector \(x\) in vector space \(V\) can be written in only one way as a linear combination of basis vectors.
Suppose the set \(\mathcal{B} =\{b_1, \ldots, b_n\}\) is an ordered basis for a subspace \(H\). For each \(x\) in \(H\), the coordinate of \(x\) relative to the basis \(\mathcal{B}\) are the weights \(c_1, \ldots, c_n\) such that \(x =c_1 b_1+ \cdots + c_n b_n\), and the vector in \(\mathbb{R}^n\)
\[[x]_{\mathcal{B}} = \begin{bmatrix}c_1 \\ \vdots \\ c_n \end{bmatrix}\]
is called the coordinate vector of \(x\) (relative to \(\mathcal{B}\)) or the \(\mathcal{B}\)-coordinate vector of \(x\)
These are NOT bases for \(\mathbb{R}^2\):
The set \(\{\hat{i} \}\) containing the vector \(\hat{i}=\langle 1,0\rangle\) by itself is not a basis for \(\mathbb{R}^2\), because there are some (in fact, many!) vectors in \(\mathbb{R}^2\) which cannot be reached by a linear combination of \(\hat{i}\) alone.
Let \(h =\begin{bmatrix}2 \\ 3\end{bmatrix}\) (or pick any other vector in \(\mathbb{R}^2\) if you wish). The set \(\{h, \hat{i}, \hat{j}\}\) is not a basis for \(\mathbb{R}^2\), because there are now redundant combinations of vectors which have the same result. For example,
\[\begin{align*}
\begin{bmatrix}4 \\ 9\end{bmatrix} & = 4\hat{i} + 9\hat{j} \text{ but also}\\
& = 2h + 3\hat{j} \text{ or even} \\
& = 3h - 2\hat{i}
\end{align*}\]
A basis for a vector space allows every point in the vector space to be reached, but with only one linear combination of the basis vectors.
Converting from the alternate basis to the standard basis. Suppose
\(\mathcal{B} =\left\{ \begin{bmatrix}1 \\ 2\\0 \end{bmatrix},\begin{bmatrix} 3 \\1 \\1 \end{bmatrix},\begin{bmatrix} -1 \\ 0 \\-1 \end{bmatrix}\right\}\) and \([u]_{\mathcal{B}} = \begin{bmatrix} 4 \\ -1\\ 2 \end{bmatrix}\). Find \(u\) in the standard basis.
\(\mathcal{B} =\left\{ \begin{bmatrix}1 \\ 1 \end{bmatrix},\begin{bmatrix} -1 \\2 \end{bmatrix}\right\} = \{ b_1, b_2\}\) is a basis for \(\mathbb{R}^2\). The standard basis for \(\mathbb{R}^2\) is \(\varepsilon= \{e_1, e_2\}\). Write the point \(P\) that is called \(\begin{bmatrix} 2 \\ -1 \end{bmatrix}\) in the standard basis in terms of basis \(\mathcal{B}\)
Notation: \(Q = \begin{bmatrix} 2 \\ -3 \end{bmatrix}_\mathcal{B} = (2) b_1 + (-3) b_2 = (2) \begin{bmatrix}1 \\ 1 \end{bmatrix} + (-3)\begin{bmatrix} -1 \\2 \end{bmatrix} =\begin{bmatrix} 5 \\-4 \end{bmatrix}_\varepsilon\)
Converting from the standard basis to an alternate basis. Suppose
\(\mathcal{B} =\left\{ \begin{bmatrix}1 \\ 3 \end{bmatrix},\begin{bmatrix} -1 \\1 \end{bmatrix}\right\}\) and \(u = \begin{bmatrix} 4 \\ 8 \end{bmatrix}\). Find \([u]_{\mathcal{B}}\).
Converting from the standard basis to a basis for a subspace. Suppose
\(\mathcal{B} =\left\{ \begin{bmatrix} 3\\6\\2 \end{bmatrix},\begin{bmatrix} -1\\0\\1 \end{bmatrix} \right\}\) and \(x = \begin{bmatrix} 3 \\ 12 \\7 \end{bmatrix}\). Determine if \(x\) is in the plane spanned by \(\mathcal{B}\), and if so, find \([x]_{\mathcal{B}}\)
4.5 Dimension of a Vector Space
Dimension
Let \(S\) be a subspace of \(\mathbb{R}^n\) for some \(n\), and \(\mathcal{B}\) be a basis for \(S\). The dimension of \(S\) is the number of vectors in \(\mathcal{B}\).
Find the dimensions of the null space and the column space of
\[B =\begin{bmatrix} 1&4&0&2&-1 \\ 3&12&1&5&5 \\ 2&8&1&3&2 \\ 5&20&2&8&8 \end{bmatrix}\]
Find the dimensions of the null space and the column space of
\[A =\begin{bmatrix} -3 & 6 &-1 &1 &-7 \\ 1& -2& 2 &3 &-1 \\ 2& -4& 5& 8 &-4 \end{bmatrix}\]
(You do not have to find the basis vectors, just the dimensions.)
Find a basis and state the dimension of the subspace: \(\left\{ \begin{bmatrix} ~a+b~ \\2a \\3a-b \\-b \end{bmatrix} ~:~ a, b \in \mathbb{R} \right\}\)
\(V\) is vector space. Mark each statement TRUE or FALSE.
The number of pivot columns of a matrix equals the dimension of its column space.
a plane in \(\mathbb{R}^3\) is a two-dimensional subspace of \(\mathbb{R}^3\).
If dim \(V\) = \(n\) and \(S\) is a linearly independent set in \(V\), then \(S\) is a basis for \(V\).
If a set \(\{ v_1, \ldots , v_p\}\) spans a finite-dimensional vector space \(V\) and if \(T\) is a set of more than \(p\) vectors in \(V\), then \(T\) must be linearly dependent.
\(\mathbb{R}^2\) is a two-dimensional subspace of \(\mathbb{R}^3\).
The number of variables in the equation \(ax = 0\) equals the dimension of Nul \(A\).
A vector space is infinite-dimensional if it is spanned by an infinite set.
If dim \(V\) = \(n\) and if \(S\) spans \(V\), then \(S\) is a basis of \(V\).
The only three-dimensional subspace of \(\mathbb{R}^3\) is \(\mathbb{R}^3\) itself.
Rank
The rank of a matrix \(A\), denoted rank \(A\), is the dimension of the column space of \(A\).
If a matrix \(A\) has \(n\) columns, then
\[\text{rank } A + \dim(\text{Nul } A) = n\]
- What is the minimum rank of a \(5 \times 7\) matrix?
- What is the maximum rank of a \(5 \times 7\) matrix?
- What is the minimum rank of a \(7 \times 5\) matrix?
- What is the maximum rank of a \(7 \times 5\) matrix?
- If a \(7 \times 7\) matrix is invertible, what is its rank?
- If the subspace of all solution of \(Ax=0\) has a basis consisting of three vectors and if \(A\) is a \(5\times 7\) matrix, what is the rank of \(A\)?
- What is the rank of a \(4 \times 5\) matrix whose null space is three-dimensional?
- If the rank of a \(7 \times 6\) matrix \(A\) is 4, what is the dimension of the solution space of \(Ax=0\)?
Theorem [The Invertible Matrix Theorem (continued)]
Let \(A\) be an \(n \times n\) matrix. Then the following statements are equivalent to the statements found in the Invertible Matrix Theorem given in Chapter 2 (including the statement that A is invertible):
The columns of \(A\) form a basis for \(\mathbb{R}^n\).
Col \(A = \mathbb{R}^n\).
dim Col \(A\) = \(n\).
rank \(A\) = \(n\).
Nul \(A\) = {0}.
dim Nul \(A\) = 0.
\(A\) is an \(m \times n\) matrix. Mark each statement TRUE or FALSE.
- The row space of \(A\) is the same as the column space of \(A^T\).
- If \(B\) is any echelon form of \(A\), and if \(B\) has three nonzero rows, then the first three rows of \(A\) form a basis for Row \(A\).
- The dimensions of the row space and the column space of \(A\) are the same, even if \(A\) is not square.
- If \(B\) is any echelon form of \(A\), then the pivot columns of \(B\) form a basis for the column space of \(A\).
- Row operations preserve the linear dependence relations among the rows of \(A\).
- The dimension of the null space of \(A\) is the number of columns of \(A\) that are not pivot columns.
- The row space of \(A^T\) is the same as the column space of \(A\).
- If \(A\) and \(B\) are row equivalent, then their row spaces are the same.
4.6 Change of Basis
In section 4.4 we saw how if we had an ordered basis \(\mathcal{B}=\{b_1, \ldots, b_n\}\) then we could write vectors either in the basis \(\mathcal{B}\) or in the standard basis. The physical location of the point is the same but the way of describing it will depend on the basis. In the standard basis we usually just call the point \(x\). In any other basis we need to specify: \([x]_\mathcal{B}\). It is reasonably straight forward to convert from the alternate basis to the standard basis and a bit trickier converting from standard to an alternate basis. What is most difficult is converting between two non-standard bases.
Let \(\mathcal{B}=\{b_1, b_2\}\) and \(\mathcal{C}=\{c_1, c_2\}\) be bases for a vector space \(V\), and suppose:
\[b_1 = 8c_1-2c_2 \qquad \text{and} \qquad b_2 = -9c_1 + 4c_2\]
Find \([x]_\mathcal{C}\) if you know that \(x = -3b_1 + 6b_2\). Another way to say that is that \([x]_\mathcal{B} = \begin{bmatrix} -3 \\ 6 \end{bmatrix}\).
Solution: To find \([x]_\mathcal{C}\) we will convert each of our basis elements to \(\mathcal{C}\) to get the following:
\[[x]_\mathcal{C} = -3[b_1]_\mathcal{C}+6 [b_2]_\mathcal{C}\]
We can write this as a matrix:
\[[x]_\mathcal{C} = \left[[b_1]_\mathcal{C} \ [b_2]_\mathcal{C}\right]\begin{bmatrix} -3 \\ 6 \end{bmatrix}\]
We are told the relationship between the elements of the basis. We know that \(b_1 = 8c_1-2c_2\) and \(b_2 = -9c_1 + 4c_2\) so:
\[[b_1]_\mathcal{C} = \begin{bmatrix} 8 \\ -2 \end{bmatrix} \qquad \text{and} \qquad [b_2]_\mathcal{C} = \begin{bmatrix} -9 \\ 4 \end{bmatrix}\]
Now we use the equation:
\[[x]_\mathcal{C} = \left[[b_1]_\mathcal{C} \ [b_2]_\mathcal{C}\right]\begin{bmatrix} -3 \\ 6 \end{bmatrix}\]
\[[x]_\mathcal{C} =\begin{bmatrix} 8 & -9 \\ -2 & 4 \end{bmatrix} \begin{bmatrix} -3 \\ 6 \end{bmatrix} = \begin{bmatrix} -78 \\ 30 \end{bmatrix}\]
The matrix \(\left[[b_1]_\mathcal{C} \ [b_2]_\mathcal{C}\right]\) is known as the change-of-coordinates matrix from \(\mathcal{B}\) to \(\mathcal{C}\).
Let \(\mathcal{B}=\{b_1, \ldots, b_n\}\) and \(\mathcal{C}=\{c_1, \ldots, c_n\}\) be bases for a vector space \(V\). Then there is a unique \(n \times n\) matrix \(\underset{\mathcal{C}\leftarrow \mathcal{B}}{P}\) such that
\[[x]_\mathcal{C} = \underset{\mathcal{C}\leftarrow \mathcal{B}}{P} [x]_\mathcal{B}\]
The columns of \(\underset{\mathcal{C}\leftarrow \mathcal{B}}{P}\) are the \(\mathcal{C}\)-coordinate vectors of the vectors in the basis \(\mathcal{B}\).
\[\underset{\mathcal{C}\leftarrow \mathcal{B}}{P} = \begin{bmatrix} [b_1]_\mathcal{C} & [b_2]_\mathcal{C} &\ldots & [b_n]_\mathcal{C}\end{bmatrix}\]
\(\underset{\mathcal{C}\leftarrow \mathcal{B}}{P}\) is called the change-of-coordinates matrix from \(\mathcal{B}\) to \(\mathcal{C}\)
Let \(A = \{a_1, a_2, a_3\}\) and \(D = \{d_1, d_2, d_3\}\) be bases for \(V\), and let
\(P= \begin{bmatrix} [d_1]_A & [d_2]_A & [d_3]_A\end{bmatrix}\).
- Which direction does this matrix transform the coordinates? \(A \rightarrow D\) or \(D \rightarrow A\).
- Write an expression to convert \([x]_D\) into \([x]_A\). (Ans. \([x]_A = P[x]_D\))
Let \(A = \{a_1, a_2, a_3\}\) and \(D = \{d_1, d_2, d_3\}\) be bases for \(V\), and suppose:
\[a_1 = 4d_1 -d_2, \ a_2 = -d_1 + 3d_2+d_3, \ a_3 = d_2 - 5d_3\]
- Find the change-of-coordinates matrix from \(A \rightarrow D\).
- Find \([x]_D\) for \(x = 5a_1 +6a_2 +a_3\) (ans \([x]_D=\begin{bmatrix} 14\\14\\1\end{bmatrix}\))
Let \(\mathcal{B}=\{b_1, b_2\}\) and \(\mathcal{C}=\{c_1, c_2\}\) be bases for \(\mathbb{R}^2\). Find the change of coordinates matrix from \(\mathcal{B}\) to \(\mathcal{C}\) knowing:
\[b_1 = \begin{bmatrix} -1 \\ 5 \end{bmatrix}, \quad b_2 = \begin{bmatrix} 1 \\ 4 \end{bmatrix}, \quad c_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \quad c_2 = \begin{bmatrix} 1 \\ 1\end{bmatrix}\]
Solution: To convert from \(\mathcal{B}\) to \(\mathcal{C}\) we need to find \([b_1]_\mathcal{C} =\begin{bmatrix} x_1 \\x_2 \end{bmatrix}\) and \([b_2]_\mathcal{C} =\begin{bmatrix} y_1 \\y_2 \end{bmatrix}\). So we solve the equations:
\[[c_1 \ c_2]\begin{bmatrix} x_1 \\x_2 \end{bmatrix} = b_1 \qquad \text{and} \qquad [c_1 \ c_2]\begin{bmatrix} y_1 \\ y_2 \end{bmatrix}=b_2\]
This can be done in a single matrix \([c_1 \ c_2 \ | \ b_1 \ b_2]\) which will row reduce to
\[\begin{bmatrix} I \ |\ \underset{\mathcal{C}\leftarrow \mathcal{B}}{P}\end{bmatrix}\]
