Chapter 1 - Linear Equations in Linear Algebra
1.4 - The Matrix Equations Ax = b
Results
A fundamental idea in linear algebra is to view linear combinations of vectors
as the product of a matrix and a vector.
Definition
If A is a $m\times n$ matrix with columns a1,...,an
and if $\bs{x}\in\R^n$, then the product of x and $A$, denoted by $A\bs{x}$,
is the linear combination of the columns of $A$ using the corresponding entries
in x as weights. That is:
$$
A\bs{x}
\;=\;
\big[\;\bs{a}_1 \;\; \bs{a}_2 \;\; \cdots \;\; \bs{a}_1 \; \big]
\begin{bmatrix*}[r]
x_1 \\
\vdots \\
x_n
\end{bmatrix*}
\;=\;
x_1\bs{a}_1 + x_2\bs{a}_2 + \ldots + x_n\bs{a}_n
$$
(Note that this is only defined if the number of columns in A correspond to the
number of elements/rows in
x).
Example
Calculating results in two different ways. Linear combination way:
$$
\begin{bmatrix*}[rrr]
1 & 2 & -1 \\
0 & -5 & 3
\end{bmatrix*}
\begin{bmatrix*}[r]
4 \\
3 \\
7
\end{bmatrix*}
\;=\;
4
\begin{bmatrix*}[r]
1 \\
0
\end{bmatrix*}
+
3
\begin{bmatrix*}[r]
2 \\
-5
\end{bmatrix*}
+
7
\begin{bmatrix*}[r]
-1 \\
3
\end{bmatrix*}
\;=\;
\begin{bmatrix*}[r]
4 \\
0
\end{bmatrix*}
+
\begin{bmatrix*}[r]
6 \\
-15
\end{bmatrix*}
+
\begin{bmatrix*}[r]
-7 \\
21
\end{bmatrix*}
\;=\;
\begin{bmatrix*}[r]
3 \\
6
\end{bmatrix*}
$$
With the "row-vector rule":
$$
\begin{bmatrix*}[rrr]
1 & 2 & -1 \\
0 & -5 & 3
\end{bmatrix*}
\begin{bmatrix*}[r]
4 \\
3 \\
7
\end{bmatrix*}
\;=\;
\begin{bmatrix*}[c]
1(4) + 2(3) + (-1)(7) \\
0(4) + (-5)(3) + 3(7)
\end{bmatrix*}
\;=\;
\begin{bmatrix*}[c]
4 + 6 -7 \\
0 - 15 + 21
\end{bmatrix*}
\;=\;
\begin{bmatrix*}[c]
3 \\
6
\end{bmatrix*}
$$
We can express a linear system as a
matrix equation. Take the linear system:
$$
\begin{array}{rcccrcr}
x_1 & + & 2x_2 & - & x_3 & = & 4 \\
& - & 5x_2 & + & 3x_3 & = & 1
\end{array}
$$
We can write this as:
$$
x_1\begin{bmatrix*}[r]
1 \\
0
\end{bmatrix*}
+
x_2\begin{bmatrix*}[r]
2 \\
-5
\end{bmatrix*}
+
x_3\begin{bmatrix*}[r]
-1 \\
3
\end{bmatrix*}
=
\begin{bmatrix*}[r]
4 \\
1
\end{bmatrix*}
$$
And this can be re-expressed as a matrix equation on the form $A\bs{x} = \bs{b}$:
$$
\begin{bmatrix*}[rrr]
1 & 2 & -1 \\
0 & -5 & 3
\end{bmatrix*}
\begin{bmatrix*}[r]
x_1 \\
x_2 \\
x_3
\end{bmatrix*}
\;=\;
\begin{bmatrix*}[r]
4 \\
1
\end{bmatrix*}
$$
Theorem 1.3
If A is an m×n matrix with columns a1, ..., an
and if b is in $\R^n$, the matrix equation
$$
A\bs{x} = \bs{b}
$$
has the same solution set as the vector equation
$$
x_1\bs{a}_1 + x_2\bs{a}_2 + \ldots + x_n\bs{a}_n = \bs{b}
$$
which, in turn, has the same solutuion set as the system of linear equations
whose augmented matrix is:
$$
\big[\;\bs{a}_1 \;\; \bs{a}_2 \;\; \ldots \;\; \bs{a}_n \;\; \bs{b} \;\big]
$$
Any problem can be viewed in three different ways: as a matrix equation,
as a vector equation, or as a system of linear equations.
Existence of Solutions
Let's begin with a stated fact.
Fact
The equation $A\bs{x} = \bs{b}$ has a solution if and only if b is
a linear combination of the columns of A.
When we say that the columns of $A$ span $\R^m$, we mean that every element in
$\R^m$ is a linear combination of the columns of $A$. Note that the following
theorem only applies to the coefficient matrix (not the augmented matrix).
Theorem 1.4
Let A be an m×n matrix. Then the following statements are logically
equivalent. That is, for a particular A, either they are all true or they
are all false.
- For each b in $\R^m$, the equation $A\bs{x} = \bs{b}$
has a solution.
- Each b in $\R^m$ is a linear combination of the columns in A
- The columns of A span $\R^m$
- A has a pivot position in every row
Properties of the Matrix-Vector Product Ax
These facts will be important throughout the book.
Theorem 1.5
If A is an m×n matrix,
u and
v are vectors in $\R^n$ and
$c$ is a scalar, then:
- $A(\bs{u} + \bs{v}) = A\bs{u} + A\bs{v}$
- $A(c\bs{u}) = c(A\bs{u})$
Exercise 3
Calculate the product.
$$
\begin{bmatrix*}[rr]
6 & 5 \\
-4 & -3 \\
7 & 6
\end{bmatrix*}
\begin{bmatrix*}[r]
2 \\
-3
\end{bmatrix*}
$$
Answer
$$
\begin{align*}
A\bs{x} &=
\begin{bmatrix*}[rcr]
6(2) & + & 5(-3) \\
-4(2) & + & (-3)(-3) \\
7(2) & + & 6(-3)
\end{bmatrix*}\\
&\\
&=
\begin{bmatrix*}[c]
12 - 15 \\
-8 + 9 \\
14 - 18
\end{bmatrix*}\\
&\\
&=
\begin{bmatrix*}[r]
-3 \\
1 \\
-4
\end{bmatrix*}
\end{align*}
$$
■
Exercise 4
Calculate the product.
$$
\begin{bmatrix*}[rrr]
8 & 3 & -4 \\
5 & 1 & 2
\end{bmatrix*}
\begin{bmatrix*}[r]
1 \\
1 \\
1
\end{bmatrix*}
$$
Answer
$$
\begin{align*}
A\bs{x} &=
\begin{bmatrix*}[rcrcr]
8(1) &+& 3(1) &+& (-4)(1) \\
5(1) &+& 1(1) &+& 2(1)
\end{bmatrix*}\\
&\\
&=
\begin{bmatrix*}[c]
8 + 3 - 4 \\
5 + 1 + 2
\end{bmatrix*}\\
&\\
&=
\begin{bmatrix*}[r]
7 \\
8
\end{bmatrix*}
\end{align*}
$$
■
Exercise 13
Is
u in the plane spanned by the columns of A?
$$
\bs{u} =
\begin{bmatrix*}[r]
0 \\
4 \\
4
\end{bmatrix*},
\qquad
A =
\begin{bmatrix*}[rr]
3 & -5 \\
-2 & 6 \\
1 & 1
\end{bmatrix*}
$$
Answer
Represent this as an augmented matrix. We will check if
u is a linear combination
of the columns in A.
$$
\begin{bmatrix*}[rrr]
3 & -5 & 0\\
-2 & 6 & 4 \\
1 & 1 & 4
\end{bmatrix*}
$$
Removing 3 and -2 from column 1. I - 3III, II + 2II
$$
\begin{bmatrix*}[rrr]
0 & -8 & -12\\
0 & 8 & 12 \\
1 & 1 & 4
\end{bmatrix*}
$$
Replacing I and III.
$$
\begin{bmatrix*}[rrr]
1 & 1 & 4\\
0 & 8 & 12 \\
0 & -8 & -12
\end{bmatrix*}
$$
III + II
$$
\begin{bmatrix*}[rrr]
1 & 1 & 4\\
0 & 8 & 12 \\
0 & 0 & 0
\end{bmatrix*}
$$
(1/8)II
$$
\begin{bmatrix*}[rrr]
1 & 1 & 4\\
0 & 1 & 3/2 \\
0 & 0 & 0
\end{bmatrix*}
$$
I - II
$$
\begin{bmatrix*}[rrr]
1 & 0 & 5/2\\
0 & 1 & 3/2 \\
0 & 0 & 0
\end{bmatrix*}
$$
Here we can see that
u is in fact a linear combination of the columns in A
and is therefore included in the plane spanned by those columns.
Verifying:
$$
\left(\frac{5}{2}\right)
\begin{bmatrix*}[r]
3 \\
-2 \\
1
\end{bmatrix*}
+
\left(\frac{3}{2}\right)
\begin{bmatrix*}[r]
-5 \\
6 \\
1
\end{bmatrix*}
\;=\;
\begin{bmatrix*}[r]
15/2 \\
-5 \\
5/2
\end{bmatrix*}
+
\begin{bmatrix*}[r]
-15/2 \\
9 \\
3/2
\end{bmatrix*}
\;=\;
\begin{bmatrix*}[r]
0 \\
4 \\
4
\end{bmatrix*}
$$
■
Exercise 15
Show that the equation A
x =
b does not have a solution for all possible
b
when:
$$
A =
\begin{bmatrix*}[rr]
2 & -1 \\
-6 & 3
\end{bmatrix*},
\qquad
\bs{b} =
\begin{bmatrix*}[r]
b_1 \\
b_2
\end{bmatrix*}
$$
Writing the augmented matrix and finding the RREF.
$$
\begin{bmatrix*}[rrr]
2 & -1 & b_1\\
-6 & 3 & b_2
\end{bmatrix*}
$$
II + 3I
$$
\begin{bmatrix*}[rrc]
2 & -1 & b_1\\
0 & 0 & b_2 + 3b_1
\end{bmatrix*}
$$
(1/2)I
$$
\begin{bmatrix*}[rrc]
1 & -1/2 & b_1/2\\
0 & 0 & b_2 + 3b_1
\end{bmatrix*}
$$
This system is consistent if $b_2 + 3b_1 \not= 0$ or in another way: $b_2 \not= -3b_1$.
■
Exercise 23
Verifying statements. Answer True or False and justify the answer.
(a)
The equation Ax = b is referred to as a vector equation.
Answer: False. This is the matrix equation form.
(b)
A vector b is a linear combination of the columns of a matrix A
if and only if the equation Ax = b has at least one solution.
Answer: True. If
b is not a linear combination of the columns in A, the
system has no solution. If it is consistent, there are one or an infinite amount of
solutions.
(c)
The equation Ax = b is consistent if every augmented matrix
$[A \; \bs{b}]$ has a pivot position in every row.
Answer: False. This statement would be true for the coefficient matrix. If the augmented
matrix has a pivot position in every row, it is inconsistent.
(d)
The first entry of the product Ax is a sum of products.
Answer: True. This follows from the matrix multiplication methods.
(e)
If the columns of an m×n matrix A span $\R^m$, then the equation
Ax = b is consistent for every b in $\R^m$.
Answer: True. By Theorem 1.4. It has a pivot position in each row.
(f)
If A is an m×n matrix and if the equation Ax = b
is inconsistent for some b in $\R^m$, then A cannot have a pivot position
in every row.
Answer: True. Follows from Theorem 1.4.
■
Exercise 24
Verifying statements. Answer True or False and justify the answer.
(a)
Every matrix equation Ax = b corresponds to a vector
equation with the same solution set.
Answer: True. By Theorem 1.3.
(b)
Any linear combination of vectors can always be written in the form
Ax for a suitable matrix A and vector b.
Answer: True. By Theorem 1.3.
(c)
The solution set of a linear system whose augmented matrix is
[a1 a2 a3 b]
is the same as the solution set of Ax = b, if
A = [a1 a2 a3].
Answer: True. By Theorem 1.3.
(d)
If the equation Ax = b is inconsistent, then b is not in
the set spanned by the columns of A.
Answer: True. By Theorem 1.4.
(e)
If the augmented matrix [A b] has a pivot position in every row,
then the equation Ax = b is inconsistent.
Answer: True. A consistent system has a pivot position in every row in the coefficient matrix.
(f)
If A is an m×n matrix whose columns do not span $\R^m$, then the equation
Ax = b is inconsistent for some b in $\R^m$.
Answer: True. Example in $\R^3$: the matrix spans the x-y plane and
b has a positive z value.
■
Exercise 35
Let A be a 3×4 matrix, let
y1 and
y2
be vectors in ℝ
3, and let
w =
y1 +
y2.
Suppose
y1 = A
x1 and
y2 = A
x2
for some vectors in ℝ
4. What fact allows you to conclude that the system
A
x =
w is consistent?
Answer
A
x =
w is consistent if there exists some vector in ℝ
4 which
satisfies the equation. We are given that A
x1 =
y1
and A
x2 =
y2, which are consistent because we have
the vectors
x1,
x2 in ℝ
4.
Observe that by Theorem 1.5:
$$
A\bs{x}_1 + A\bs{x}_2 = A(\bs{x}_1 + \bs{x}_2) = A\bs{x}_s.
$$
where
xs in ℝ
4.
Now, by assumptions:
$$
A\bs{x}_1 + A\bs{x}_2 = \bs{y}_1 + \bs{y}_2 = \bs{w}.
$$
Combining these, we get:
$$
A\bs{x}_s = \bs{w}
$$
which is consistent because we know we have at least one solution:
xs.
■