Definitions & rules

A matrix is an ordered rectangular array of quantities. An array $A$ with $m$ rows and $n$ columns is called an $m \times n$ matrix and is said to have $m n$ elements. For example,

The position of an element in a matrix is specified uniquely by means of a double subscript. We denote the element in the $i$-th row and the $j$-th column of the matrix $A$ by $a_{i j}$. A shorthand notation is to write $A=\left(a_{i j}\right)_{m \times n}$ where $1 \leq i \leq m$ and $1 \leq j \leq n$.

A matrix with $m=1$ is called a row vector, for example $A=\left(\begin{array}{lll}1 & 3 & 4\end{array}\right)$ and a matrix with $n=1$ is called a column vector, for example $B=\left(\begin{array}{l}3 \\ 5 \\ 1\end{array}\right)$.

Rules of matrix algebra

I. Addition and subtraction

Consider a matrix $A=\left(a_{i j}\right)_{m \times n}$ of size $m \times n$ and a matrix $B=\left(b_{i j}\right)_{p \times q}$ of size $p \times q$. Then, $A \pm B$ can only exist if $p=m$ and $q=n$, i.e. only if $A$ and $B$ have the same size. For example, determine the matrix $A+B$ if

The resulting matrix has the same size as $A$ and $B$; if the entries for the matrix $A$ are $a_{i j}$ and for $B$ are $b_{i j}$, then the entries in $A+B$ are given by $a_{i j}+b_{i j}$, as follows,

II. Equality

The matrices $A$ and $B$ are equal to each other, i.e. $A=B$, iff $a_{i j}=b_{i j}$ for all $i$ and $j$.

III. Multiplication by a scalar

If $\lambda$ is a scalar, then $\lambda A=\lambda a_{i j}$ i.e. every element of $A$ is multiplied by $\lambda$.

IV. Matrix multiplication

If $A$ is a row vector with $n$ elements given by

and $B$ is a column vector with $n$ elements given by

then the product, $A B$ is defined to be a scalar given by

In general, if $A=\left(a_{i j}\right)_{m \times n}$ and $B=\left(b_{i j}\right)_{p \times q}$, the product $A B$ exists only if $n=p$, i.e. only if

the number of columns of $A=$ the number of rows of $B$

If $n=p$ then the product $A B$ (let us call it $C$ ) is an $m \times q$ matrix:,

where $n=p$ in Eq. (10.4). For $n=p$, we define the product $C=A B$ as a matrix of size $m \times q$ where,

Consider the matrices given by $A$ and $B$ as follows,

Suppose we want to determine the product $A B$. The number of columns of $A$ are equal to the number of rows of $B$ therefore we can proceed to determine the product $A B$. Since $A$ is a $3 \times 2$ matrix and $B$ is $2 \times 3$ matrix, the product, $A B$ will be a $3 \times 3$ matrix.

To obtain the product $A B$ : - We multiply the first element in the first row of $A$ (i.e $a_{11}=0$ ) with the first element in the first column of $B$ (i.e. $b_{11}=1$ ) [these elements are shown in blue boxes]. We then multiply the second element in the first row of $A$ (i.e. $a_{12}=4$ ) with the second element in the first column of $B$ (i.e. $b_{21}=1$ ) [these elements are shown in red boxes]. The sum of these products is the first element in the product $A B$ :

• So far we multiplied the first row of $A$ by the first column of $B$. The sum is $0+4=4$ shown in green. Next, we multiply the first row of $A$ by the second column of $B$ :

• To complete the first row of the product $A B$, we multiply the first row of $A$ by the third column of $B$ :

• Repeating the above steps with the second and third rows of $A$, yields,

Note that if $A B=0$, it does not necessarily follow that $A=0$ or $B=0$ or $B A=0$.

Properties of matrix multiplication

(i) Non-commutativity: in general, $A B \neq B A$; even if both $A B$ and $B A$ exist. If $A B=B A$ then $A$ and $B$ are said to commute.

(ii) Associativity: for matrices $A(m \times n), B(n \times q)$, and $C(q \times s)$, we have

(iii) Distributive over matrix addition: For $A(m \times n), B(n \times q)$, and $C(n \times q)$, we have

Special types of matrices

In this section, we take $A$ to be an $m \times n$ matrix, i.e. $A=\left(a_{i j}\right)_{m \times n}$.

Transpose of a matrix

Consider an $m \times n$ matrix $A$. The transpose of $A$, denoted by $A^{\top}$ is the $n \times m$ (note the index letters have been switched) matrix whose rows are the columns of $A$ and whose columns are the rows of $A$, i.e. $A=\left(a_{j i}\right)_{n \times m}$. For example, if

then,

Note that the transpose of a matrix $A$ is also denoted as $A^{t}$ or $A^{\prime}$.

I. Square matrices

If $m=n$, then $A$ is called the square matrix of order $n$ or an $n$-th order matrix. The elements $a_{i i}(i=1,2,3, \ldots, n)$ are called the main diagonal elements of $A$. Their sum is called the trace of $A$, i.e.

II. Symmetric matrices

If $A^{\top}=A$, i.e. $a_{j i}=a_{i j}$ then $A$ is said to be a symmetric matrix. For example,

III. Skew-symmetric or anti-symmetric matrices

If $A^{\top}=-A$, i.e. $a_{j i}=-a_{i j}$ then $A$ is said to be a skew or anti-symmetric matrix. For example,

It follows that $a_{i i}=0$ for such matrices.

Note that any square matrix can be written as the sum of a symmetric and an anti-symmetric matrix:

IV. Diagonal matrices

If $a_{i j}=0$ where $i \neq j, A$ is called a diagonal matrix. For example,

Special cases of diagonal matrices include the identity and null matrices. If $a_{i j}=0$ where $i \neq j$ and $a_{i i}=1$, then $A=I_{n}$ and is referred to as the identity matrix of order $n$ (note that throughout these notes we also denote the identity matrix simply by $I)$. For example,

If $a_{i j}=0$ for all $i$ and $j$ then $A=0$ and is called the null or zero matrix.

V. Lower triangular matrix:

This is a matrix for which $a_{i j}=0$ if $i<j$, e.g.

VI. Upper triangular matrix:

This is a matrix for which $a_{i j}=0$ if $i>j$, e.g.

Inverse matrices

A square matrix $A$ of order $n$ has an inverse, say $B$, if,

• If the inverse of $A$ exists, then it is unique;
• If $B$ exists, $A$ is called a non-singular matrix;
• If $B$ does not exist, $A$ is said to be singular.

The inverse of $A$ is denoted by $A^{-1}$ so,

Shortcut for $2 \times 2$ matrices

Given a $2 \times 2$ matrix,

let

where the elements $b_{i j}$ are unknown. From Eqs. (10.14), we have

and,

Equations (10.15) and (10.16) give us 8 scalar equations for the 4 unknowns $\left(b_{11}, b_{12}, b_{21}\right.$, $b_{22}$ ). It can be found that there is a unique solution for $b_{i j}$ such that:

Note that:

• the denominator in the first term on the RHS of (10.17) is the determinant of $A$ (see Section 10.2);
• the second term is the matrix $A$ with the position of the diagonals $a_{11}$ and $a_{22}$ switched while the positions of $a_{12}$ and $a_{21}$ remain the same but the elements are multiplied by -1 .

From (10.17), we can say that:

• $A$ has an inverse (i.e. it is non-singular) if $a_{11} a_{22}-a_{12} a_{21} \neq 0$;
• $A$ is singular if $a_{11} a_{22}-a_{12} a_{21}=0$. Example 10.1 If it exists, compute the inverse of the following $2 \times 2$ matrix

Solution We first calculate the denominator of the fraction in Eq. (10.17),

We then switch the entries in the diagonals and multiply the entries in the off-diagonal by -1 so that we obtain the inverse as

It is always a good idea to check your answer using $A A^{-1}=I_{2}$,

Note that to find the inverse of a matrix higher than $n=2$, we need knowledge of determinants, the matrix of minors, and the matrix of cofactors so we discuss inverses of larger matrices later in this chapter.

Orthogonal matrices

A square matrix $A$ with real entries and satisfying the condition $A^{-1}=A^{\top}$ is said to be orthogonal. Since computing the matrix inverse can be difficult while the transpose is straightforward, orthogonal matrices make a difficult operation easier. If $A A^{\top}=A^{\top} A=I$, then $A$ is orthogonal. Consider a matrix $A$ and its transpose $A^{\top}$,

Consider now the dot product of the $i^{\text {th }}$ and $j^{\text {th }}$ row vectors of $A$ [refer to Eq. (10.18)]; this is given by $\sum_{k=1}^{n} a_{i k} a_{j k}$. For the product $A A^{\top}$ to be equal to the identity matrix, we deduce the following:

Denoting the $i^{\text {th }}$ row vector of $A$ by $\boldsymbol{r}_{i}$, we have

i.e. for $i \neq j, \boldsymbol{r}_{i}$ and $\boldsymbol{r}_{j}$ must be perpendicular or orthogonal and $\left|\boldsymbol{r}_{i}\right|=1$. If any two vectors in a set $\left\{\boldsymbol{r}_{i}\right\}$ are orthogonal for all $i \neq j$ then they are said to be mutually orthogonal; further since $\left|\boldsymbol{r}_{i}\right|=1$, the row vectors of $A$ are mutually orthogonal unit vectors. Note that we also need $A^{\top} A=I$ which implies that the column vectors of $A$ must also be mutually orthogonal unit vectors.

Example 10.2 Show that the following matrix is orthogonal,

Solution The matrix $A$ is orthogonal iff Eq. (10.19) and equivalently Eq. (10.20) holds true. We define $\boldsymbol{r}_{1}=(\cos \theta,-\sin \theta, 0), \boldsymbol{r}_{2}=(\sin \theta, \cos \theta, 0)$, and $\boldsymbol{r}_{3}=(0,0,1)$. Then,

and

$\boldsymbol{r}_{1} \cdot \boldsymbol{r}_{2}=\cos \theta \sin \theta-\cos \theta \sin \theta=0, \quad \boldsymbol{r}_{1} \cdot \boldsymbol{r}_{3}=0, \quad \boldsymbol{r}_{2} \cdot \boldsymbol{r}_{3}=\sin \theta \cos \theta-\sin \theta \cos \theta=0$

Recall the dot product is commutative so the above proves that the rows of $A$ are mutually orthogonal vectors and therefore $A$ is an orthogonal matrix. It follows that,

Exercises

1. Find both $A B$ and $A+B$ of the following where possible: (a) $A=\left(\begin{array}{ll}1 & 2 \\ 2 & 1\end{array}\right)$ and $B=\left(\begin{array}{cc}1 & 1 \\ -1 & 1\end{array}\right)$ (b) $A=\left(\begin{array}{cc}1 & 2 \\ 3 & -1 \\ 2 & -1\end{array}\right)$ and $B=\left(\begin{array}{cc}2 & -2 \\ 1 & 3\end{array}\right)$; (c) $A=\left(\begin{array}{ccc}1 & -1 & 0 \\ -1 & 2 & 1 \\ 3 & -1 & 0\end{array}\right)$ and $B=\left(\begin{array}{ccc}0 & 1 & 0 \\ 0 & 2 & 2 \\ -1 & 0 & -1\end{array}\right)$.
2. Find decomposition of $A$ into symmetric and skew symmetric parts where,

3. Find the inverse of the following $2 \times 2$ matrices: (a) $A=\left(\begin{array}{cc}1 & 2 \\ -2 & 1\end{array}\right)$ (b) $A=\left(\begin{array}{cc}3 & -1 \\ 1 & 2\end{array}\right)$.