Linear algebraic equations

In this section we use our knowledge of matrices and vectors to solve simultaneous linear equations. Consider $n$ linear equations,

where $a_{i j}$ for $1 \leq i, j \leq n$ and $b_{i}$ for $1 \leq i \leq n$ are given and we wish to solve for $x_{1}, x_{2}, \cdots, x_{n}$. We can find solutions by writing this system of equations in terms of matrices and vectors as follows

where $A$ is the coefficient matrix given by $A=\left(a_{i j}\right), \boldsymbol{x}=\left(x_{1}, x_{2}, \cdots, x_{n}\right)^{\top}$ and $\boldsymbol{b}=$ $\left(b_{1}, b_{2}, \cdots, b_{n}\right)^{\top}$.

Homogeneous system of equations

We first consider the case $\boldsymbol{b}=\mathbf{0}$. Then Eq. (10.30) represents a homogeneous system given by,

We identify two cases:

1. If $A$ is non-singular, then $A^{-1}$ exists and therefore

giving the single solution $\boldsymbol{x}=\mathbf{0}$. 2. If $A$ is singular, then there exist a family of infinitely many solutions for $\boldsymbol{x}$.

Note that the homogeneous system (10.31) always has the solution $\boldsymbol{x}=\mathbf{0}$; the zero solution is called the trivial solution. If any nonzero solutions exist they are called nontrivial solutions.

Let us consider a two-dimensional system given by the following equations

for some given constants $a, b, c, d$. In matrix form, this is represented by

We can proceed to solve Eqs. (10.33) through

or, alternatively, from

But $a d-b c=\operatorname{det} A$ and so we can see from Eqs. (10.35) and (10.36) that if $A$ is nonsingular, then $\operatorname{det} A \neq 0$ and so $x$ and $y$ must both be zero. However, if $\operatorname{det} A=0$ then $a d-b c=0$ and, assuming $a$ and $c$ are both nonzero, we have $d / c=b / a$. From Eqs. (10.33), we have

It follows, that if $d / c=b / a$ then the equations in (10.37) give the same information. We have two unknowns, one equation and so we have an under-determined system with infinite solutions. For instance, we can choose $y=1$ and from that we deduce [using Eqs. (10.37)] that $x=-b / a$. Further, we can set $y$ to any number while $x$ is dictated by (10.37). As a solution to Eqs. (10.33) and equivalently (10.34), we have

for arbitrary $\lambda$. Thus, we have infinitely many solutions in this case, forming a line through the origin.

We now consider a three-dimensional system,

or, in matrix form,

The first thing to do is calculate the determinant of the matrix to determine whether it is singular. To simplify the calculation we may perform row operations as done in Example 10.7. Using the row operation $\left(R_{3}-R_{1}-R_{2}\right) \rightarrow R_{3}$, obtain

it is clear from Eq. (10.41) that the determinant, by expansion along the third row, is zero and the matrix is singular. We notice that the third row of $A$ is a linear combination of rows 1 and 2 since $R_{3}=R_{1}+R_{2}$; this makes the rows linearly dependent. Informally this means that the third equation gives no further information than that obtained from the first two. We proceed with the first two equations in Eq. (10.39). Like in the 2D system we saw above, we are left with an under-determined system since we have two equations and three unknowns,

We can rewrite the system (10.42) with $z=\lambda$ for some arbitrary $\lambda$ and solve to get $y=-\lambda$ and $x=-\lambda$; the solution then becomes

for $\lambda \in \mathbb{R}$. Geometrically we have two planes containing the origin, which intersect along a line.

Nonhomogeneous system of equations

The nonhomogeneous or inhomogeneous case in Eq. (10.30) corresponds to $\boldsymbol{b} \neq 0$. Again, we have two cases:

1. If $A$ is non-singular then, $A^{-1}$ exits and the unique solution is $\boldsymbol{x}=A^{-1} \boldsymbol{b}$;
2. If $A$ is singular then, solutions may or may not exist. In that case we have an undetermined system of equations; they are called consistent if solutions exist and inconsistent if solutions do not exist.

We proceed with an example. Consider the linear system

where $c$ is some constant. This is the same matrix used in Eq. (10.40) which we know to be singular and so we want to determine for what values of $c$ solutions exist. With $A$ singular, we can deduce that one of its columns is dependent on the other two (for a square matrix, if the rows are dependent so are the columns). To solve the system, we require that $(c, 1,5)^{\top}$ is a linear combination of two of the columns of $A$. Choosing the first and last columns we obtain,

We solve the following equations for $\lambda$ and $\mu$

there give $\lambda=1$ and $\mu=1$. It follows that $c$ must satisfy

which gives $c=4$. Therefore, if $c \neq 4$, no solutions exist. If $c=4$, the first equation in Eq. (10.43) can be obtained from the second and third equation in Eq. (10.43). To proceed, we set $z=\lambda$ for arbitrary $\lambda$ and solve,

the above equations come from writing out the first two scalar equations from the system in Eq. (10.43). Equations (10.47) give $x=\lambda$, and $y=1-\lambda$. The solution is then described by the line,

for $\lambda \in \mathbb{R}$.

To solve systems of linear equations we follow the steps outlined below:

(i) Determine whether the matrix describing the system is singular or not;

(ii) If the matrix is non-singular, we may invert it (see later sections for matrix inversion as well). In the homogeneous case, the only solution is zero. In the nonhomogeneous case, we need to invert the matrix;

(iii) If the matrix is singular, then we know that at least one equation is redundant. For the nonhomogeneous case, we will need to check whether a solution exists. We may establish this by checking whether $\boldsymbol{b}$ may be written as the linear combination of two of the columns. Once this is established, we set one variable to $\lambda$, acting as the parameter and solve the equations in terms of this free parameter.