Matrices

Matrix

A matrix is a rectangular array of numbers.

Scalars

Scalars are real numbers that define a numerical quantity.

Entries

The numbers in the array are called the entries in the matrix. Each entry has a row number $m$ and a column number $n$.
The entry that occurs in row $i$ and column $j$ of a matrix $A$ is usually denoted by $(A)_{ij}$ or by $a_{ij}$.

Matrix size

The matrix size is defined by the number of rows and the number of columns.

Notation

A matrix $A$ with $m$ rows and $n$ columns has a size of $m \cdot n$.

$$ A = \begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\ a_{21}&a_{22}&\cdots&a_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{m1}&a_{m2}&\cdots&a_{mn}\end{pmatrix} = \left[ a_{ij} \right]_{m\cdot n} $$

Special case matrices

• row vector, row matrix:

  A row vector only consists of one row:

  $$ \begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1n}\end{pmatrix} $$

• column vector, column matrix

  A column vector only consists of one column

  $$ \begin{pmatrix}a_{11}\\ a_{21}\\ \vdots\\ a_{1m}\end{pmatrix} $$

• A matrix with one row and one column can be identified with its only entry:

  $$ \begin{pmatrix} a_{11} \end{pmatrix} $$

Square matrix

A square matrix is a matrix with the same number of rows and of columns. A matrix $A$ of size $n \cdot n$ is a square matrix of order $n$.

$$ A = \begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\ a_{21}&a_{22}&\cdots&a_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n1}&a_{n2}&\cdots&a_{nn}\end{pmatrix} $$

Main diagonal

For a square matrix the main diagonal is the diagonal that is made up of the entries in the following order: $a_{11}, a_{22}, ... , a_{nn}$

Remark

The row number equals the column number for the entire diagonal.

Trace

Definition

The trace is the sum of all entries of the main diagonal in a square matrix.

$$ tr(A) = a_{11} + a_{22} + ... + a_{nn}$$

Remark

There is no trace for a non square matrix.

Equality of matrices

Theorem

Two matrices are equal if and only if they have the same size and the corresponding entries are equal.

Methods

To prove that two matrices of the same size are equal we can:

• prove that corresponding entries are the same

• prove that corresponding row vectors are the same

• prove that corresponding column vectors are the same

• prove that corresponding submatrices are the same, with the same partition

Sum of matrices

For the matrices $A$ and $B$ the sum $A + B$ is the matrix obtained by adding the entries of $B$ to the corresponding entries of $A$:

$$ \left( A + B \right)_{ij} = \left( A \right)_{ij} + \left( B \right)_{ij}$$

Remark

Matrices of different sizes cannot be added.

Difference of matrices

For the matrices $A$ and $B$ the difference $A - B$ is the matrix obtained by subtracting the entries of $B$ to the corresponding entries of $A$:

$$ \left( A - B \right)_{ij} = \left( A \right)_{ij} - \left( B \right)_{ij}$$

Remark

Matrices of different sizes cannot be subtracted.

Scalar multiples

The product of a matrix $A$ and a scalar $c$ is denoted by $cA$. The matrix $cA$ is called a scalar multiple of $A$.

Consequence

Substracting a matrix $A$ from a matrix $M$ means adding the scalar multiple $c$ of -1 times the matrix $A$ to $M$.

$$ M-A = M + (-1) A $$

Linear combinations of matrices

If $A_1, . . . , A_r$ are matrices of the same size, and if $c_1, . . . , c_r$ are scalars, then an expression of the form:

$$ c_1A_1 + c_2A_2 + ... + c_rA_r $$

is a linear combination of $A_1, . . . , A_r$ with coefficients $c_1, ... , c_r$.

Conditions for multiplying matrices

For the numbers $r,m,n \in \mathbb{N^*}$: $m>r$ and $n>r$, we can multiply a $m \cdot r$ matrix by a $r \cdot n$ matrix, and the result is an $m \cdot n$ matrix.

Row column rule

Let $A$ and $B$ be 2 matrices:

$$A = \begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1r}\\ a_{21}&a_{22}&\cdots&a_{2r}\\ \vdots&\vdots&\ddots&\vdots\\ a_{i1}&a_{i2}&\cdots&a_{ir}\\ a_{m1}&a_{m2}&\cdots&a_{mr}\end{pmatrix} $$ $$ B= \begin{pmatrix}b_{11}&b_{12}&\cdots&b_{1j}&\cdots&b_{1n}\\ b_{21}&b_{22}&\cdots&b_{2j}&\cdots&b_{2n}\\ \vdots&\vdots&\ddots&\vdots & \ddots &\vdots \\ b_{r1}&b_{r2}&\cdots&b_{rj}&\cdots&b_{rn}\end{pmatrix}$$

The row column rule for matrix multiplication is:

$$ A B = \begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1r}\\ a_{21}&a_{22}&\cdots&a_{2r}\\ \vdots&\vdots&\ddots&\vdots\\ a_{i1}&a_{i2}&\cdots&a_{ir}\\ a_{m1}&a_{m2}&\cdots&a_{mr}\end{pmatrix} \begin{pmatrix}b_{11}&b_{12}&\cdots&b_{1j}&\cdots&b_{1n}\\ b_{21}&b_{22}&\cdots&b_{2j}&\cdots&b_{2n}\\ \vdots&\vdots&\ddots&\vdots & \ddots &\vdots \\ b_{r1}&b_{r2}&\cdots&b_{rj}&\cdots&b_{rn}\end{pmatrix}$$

the entry $(AB)_{ij}$ in row $i$ and column $j$ of $AB$ is given by:

$$ \left( AB \right)_{ij} =a_{i1}b_{1j}+a_{i2}b_{2j}+\ldots +a_{ir}b_{rj} $$ $$ = \sum_{h=1}^{r} a_{ih}b_{hj} $$

Remark

The rows of $A$ and the columns of $B$ must have the same length. Here: $r$.

Matrix products and linear combinations

Theorem

Let $A$ be an $m \cdot n$ matrix, and $x$ an $n \cdot 1$ column vector. The product $Ax$ is a linear combination of the column vectors of $A$, the coefficients being the entries of $x$.

$$ A = \begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\ a_{21}&a_{22}&\cdots&a_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{m1}&a_{m2}&\cdots&a_{mn}\end{pmatrix} $$ $$ x = \begin{pmatrix}x_{1}\\ x_{2}\\ \vdots\\ x_{n}\end{pmatrix}$$

Then:

$$ Ax = \begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\ a_{21}&a_{22}&\cdots&a_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{m1}&a_{m2}&\cdots&a_{mn}\end{pmatrix} \begin{pmatrix}x_{1}\\ x_{2}\\ \vdots\\ x_{n}\end{pmatrix}$$ $$=\begin{pmatrix}a_{11}x_{1}+a_{12}x_{2}+\cdots +a_{1n}x_{n}\\ a_{21}x_{1}+a_{22}x_{2}+\cdots +a_{2n}x_{n}\\ \vdots\\ a_{m1}x_{1}+a_{m2}x_{2}+\cdots +a_{mn}x_{n}\end{pmatrix}$$ $$= x_{1}\begin{pmatrix}a_{11}\\ a_{21}\\ \vdots\\ a_{m1}\end{pmatrix} +x_{2}\begin{pmatrix}a_{12}\\ a_{22}\\ \vdots\\ a_{m2}\end{pmatrix} + ... + x_n \begin{pmatrix}a_{1n}\\ a_{2n}\\ \vdots\\ a_{mn}\end{pmatrix} $$

Column-row expansion

Let $A$ be an $m \cdot s$ matrix and $B$ an $s \cdot n$ matrix. Considering the $i$^th column $cA$, $i$ of $A$ and the $i$^th row $r_{B,i}$ of $B$,

$$ AB = c_{A,1} r_{B,1} + c_{A,2} r_{B,2} + ... + c_{A,s} r_{B,s} $$

Matrix form of a linear equation

A linear system:

$$\begin{align*} \left\{ \begin{array}{ccccccccccc} a_{11}x_{1} & + & a_{12}x_{2} & + & \cdots & + & a_{1n}x_{n} & = & b_{1} \\ a_{21}x_{1} & + & a_{22}x_{2} & + & \cdots & + & a_{2n}x_{n} & = & b_{2} \\ \vdots & & \vdots & & \ddots & & \vdots & & \vdots \\ a_{m1}x_{1} & + & a_{m2}x_{2} & + & \cdots & + & a_{mn}x_{n} & = & b_{m} \\ \end{array} \right. \end{align*} $$

can be expressed as the equality of 2 column matrices:

$$ \begin{pmatrix}a_{11}x_{1}+a_{12}x_{2}+\cdots +a_{1n}x_{n}\\ a_{21}x_{1}+a_{22}x_{2}+\cdots +a_{2n}x_{n}\\ \vdots\\ a_{m1}x_{1}+a_{m2}x_{2}+\cdots +a_{mn}x_{n}\end{pmatrix} = \begin{pmatrix}b_{1}\\ b_{2}\\ \vdots\\ b_{m}\end{pmatrix}$$

we can also write:

$$ Ax = b $$

we get therefore

$$ \begin{pmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\ a_{21}&a_{22}&\cdots&a_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{m1}&a_{m2}&\cdots&a_{mn}\end{pmatrix} \begin{pmatrix}x_{1}\\ x_{2}\\ \vdots\\ x_{n}\end{pmatrix} = \begin{pmatrix}b_{1}\\ b_{2}\\ \vdots\\ b_{m}\end{pmatrix}$$

We obtain the matrix $A$ wich is called the coefficient matrix.
The augmented matrix is $[A,b]$, with $b$ as an additional last column.

Transpose

Let $A$ be an $m \cdot n$ matrix.
The transpose of $A$ is the $n \cdot m$ matrix $A^T$ obtained as follows:

• the k^th column of $A^T$ is the k^th row of $A$

• equivalently, the k^th row of $A^T$ is the k^th column of $A$

We are reversing the indices, $\left( A^T \right)_{ij} = \left( A \right)_{ji}$

Transpose of a square matrix

A square matrix $A$ and its transpose $A^T$ are symmetric with respect to the main diagonal:

• The entries on the main diagonal are the same

• The entries above the main diagonal get swapped with the entries below the main diagonal

Properties of the transpose

Provided that the sizes of the matrices are such that the stated operations can be performed:

• $ \left( A^T \right)^T = A $

• $ \left( A + B \right)^T = A^T + B^T $

• $ \left( A - B \right)^T = A^T - B^T $

• $ \left( kA \right)^T = kA^T $

• $ \left( A B \right)^T = B^T A^T $

Generalisations

• The transpose of a sum of any number of matrices is the sum of the transposes.

• The transpose of a product of any number of matrices is the product of the transposes in the reverse order.

Algebraic properties

Let $A, B, C$ be matrices, and let $a, b$ be scalars. Assuming that the sizes of the matrices are such that the indicated operations can be performed, we have:

• Commutative law for matrix addition

  $A+B = B+A$

• Associative law for matrix addition

  $A+(B+C) = (A + B) + C$

• Associative law for matrix multiplication

  $A(BC) = (A B) C$

• Left distributive law

  $A(B+C) = AB + AC$

• Right distributive law

  $(B+C)A = BA + CA$

Other properties are:

• $A(B-C) = AB -AC$

• $(B-C)A = BA-CA$

Scalar properties:

• $a(B+C) = aB + aC$

• $a(B-C) = aB - aC$

• $(a+b)C = aC + bC$

• $a(B-C) = aB - aC$

• $a(bC) = (ab)C$

• $a(BC) = (aB)C = B(aC)$

Remark

There are three reasons why the matrix product is not commutative:

1. $AB$ may be defined and $BA$ may not

2. if $AB$ and $BA$ are defined, they may have different sizes

3. if $AB$ and $BA$ are defined and have the same size, the two products are usually different

Zero matrices

Definition

A zero matrix is a matrix where every entries are zero.

Notation

The zero matrix $m \cdot n$ is commonly denoted as: $0_{m \cdot n}$ or 0.

Consequences

• The zero matix is the neutral element for addition.

  $$ A + 0 = 0 + A = 0 $$

• The zero matrix is the additive inverse of $A$ to $-A$.

  $$ A + (-A) = (-A) + A = 0 $$

Remark

There are zero matrices for every possible size.

Operations with zero

Let 0$_{m \cdot n}$ be the zero matrix of size $m \cdot n$ and let $A_{m \cdot n}$ be the matrix of size $m \cdot n$ and let $a$ be a nonzero scalar:

• Scalar times zero matrix:

  $$ a \cdot 0_{m \cdot n} = 0_{m \cdot n} $$

• Scalar 0 times matrix

  $$ 0 \cdot A = 0_{m \cdot n} $$

• Multiplying a matrix by a zero matrix

  $\forall m \cdot n$ matrices $A$ and $\forall r \geq 1$

  $$ 0_{r \cdot m} \cdot A = 0_{r \cdot n} $$ $$ A \cdot 0_{n \cdot r} = 0_{m \cdot r} $$

Zero-product

Let $c$ be a scalar, and let $A$ be a matrix:

$$ cA=0\ \Leftrightarrow c=0\ \vee \ A=0 $$

or:

$$ c\neq 0\ \wedge \ A\neq 0\ \Leftrightarrow cA\neq 0$$

Remark

This property does not hold for matrix multiplication.

Cancellation law for scalar multiplication

For scalar multiplication we have the cancellation laws:

• If $cA = cA^{\prime}$ and $c\neq 0$, then $A=A^{\prime}$

• If $cA = c^{\prime}A$ and $A\neq 0$, then $c=c^{\prime}$

Remark

This property does not hold for matrix multiplication.

Identity matrix

A square matrix with 1's on the main diagonal and zeros elsewhere is called identity matrix. There are identity matrices of any order. We write $I$ or $I_n$ for the $n \cdot n$ identity matrix.

$$ I = \begin{pmatrix}1&0&\cdots&0\\ 0&1&\cdots&0\\ \vdots&\vdots&\ddots&\vdots\\ 0&0&\cdots&1\end{pmatrix} $$

Properties

• If A is an $m \cdot n$ matrix, we have:

  $$ A I_n = A $$ $$ I_m A = A $$

• Consider $n \cdot n$ matrices. Then $I_n$ is the neutral element of the multiplication:

  $$ A I_n = I_n A = A $$

Remark

Sometimes the identity matrix $I$ is also denoted with the symbol:

$$ \mathbb{1} $$

Powers of a square matrix

Definitions

• Let $A$ be a $m \cdot m$ matrix. We define:

  $$A^0 = I_m $$

• For every integer $n \geq 1$ we define:

  $$ A^n \mathrel{\mathop:}= \underbrace{AA\ldots A}_{n\ factors} $$

Theorem

For every integer $r , s \geq 0$

$$ A^r A^s = A^{r+s} $$ $$\left( A^r \right)^s = A^{rs} $$

Remark

Powers of a same square matrix commute.

Matrix polynomials

Let $A$ be an $n \cdot n$ matrix and consider the polynomial with $x \in \mathbb{N} $ and $c\in \mathbb{R}$:

$$ p(x) = c_0 + c_1x + c_2x^2 +···+ c_mx^m $$

We define the matrix polynomial:

$$ p(A) = c_0I_n + c_1A + c_2A^2 +···+ c_mA^m $$

Remark

This polynomial is again a matrix of size $n \cdot n$.

Property

Since powers of a square matrix commute, and since a matrix polynomial in $A$ is built up from powers of $A$, any two matrix polynomials in $A$ also commute. That is, for any polynomials $p_1$ and $p_2$ we have:

$$ p_1(A)p_2(A) = p_2(A)p_1(A) $$

Inverse, Nonsingular, Nondegenerate, Regular

Let $A$ be a square matrix. If there is a square matrix $B$ of the same size such that

$$ AB = BA = I $$

then $A$ is invertible or nonsingular and $B$ is an inverse of $A$. Else, $A$ is singular.

Condition

For a matrix $A$ to be invertible the determinant of the matrix can not be 0.

$$ \det \left( A \right) \neq 0 $$

Remark

If $B$ is an inverse of $A$, then $A$ is an inverse of $B$.

Non-invertible, Singular

A non-invertible or singular matrix is a matrix that can not be inverted.

Unicity of inverses

Theorem

If a square matrix has an inverse, then the inverse is unique. If $B$ and $C$ are inverses of $A$, then $B= C$.

Proof

$$ B= BI= B (AC) $$

by Associativity:

$$ = (BA)C= IC= C $$

$\square$

Inverse and Transpose

Theorem

If $A$ is an invertible matrix, then $A^T$ is also invertible. We have:

$$ \left( A^T \right)^{-1} = \left( A^{-1} \right)^T $$

Proof

Let $A$ be a square matrix and its determinant be non-zero:

$$ A^T (A^{-1})^T = (A^{-1}A)^T = I^T = I $$

$\square$

Inverse of matrix products

Theorem

If $A$ and $B$ are invertible matrices of the same size, then $AB$ is invertible. We have:

$$ (AB)^{-1} = B^{-1}A^{-1}$$

Proof

Let $A$ and $B$ be invertible matrices:

$$(AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = AA^{-1} = I$$

$\square$

Negative powers of invertible matrices

Suppose that $A$ is invertible, $A$ is a square matrix and $n \in \mathbb{Z}$:

• We define

$$ A^{-n} := \left( A^{-1} \right)^{n} = \underbrace{A^{-1} A^{-1} \ldots A^{-1}}_{n factors} $$

• All powers of $A$ are invertible.

• For any scalar $c \neq 0$ the matrix $cA$ is invertible and the inverse is $\frac{1}{c} A^{-1}$

Inverting elementary row operations

Every elementary row operation has an inverse which are:

• Multiply a row by a nonzero constant $c$.
  Multiply the same row by $\frac{1}{c}$.

• Interchange two rows.
  Interchange the same two rows.

• Add $cr_i$ to $r_j$ $(i \neq j)$.
  Add $−cr_i$ to $r_j$.

If a matrix $B$ is obtained from a matrix $A$ by performing a sequence of elementary row operations, then there is a sequence of elementary row operations, which when applied to $B$ gives $A$.

Row equivalence

Two matrices are row equivalent if one can be obtained from the other by elementary row operations. It is an equivalence relation:

1. reflexive: $A$ is row equivalent to $A$

2. symmetric: if $A$ is row equivalent to $B$, then $B$ is row equivalent to $A$

3. transitive: if $A$ is row equivalent to $B$ and $B$ is row equivalent to $C$, then $A$ is row equivalent to $C$

Theorem

Two matrices are row equivalent if and only if they have the same reduced row echelon form.

Elementary matrix

Definition

A square matrix is called an elementary matrix if it can be obtained from the identity matrix by performing one elementary row operation.

Elementary row operation as the multiplication by an elementary matrix

Theroem

If the elementary matrix $E$ results from a certain elementary row operation on the identity matrix $I$ , performing this elementary row operation on a matrix $A$ gives $EA$.

Invertiblity of elementary matrices

Theorem

Every elementary matrix is invertible. The inverse is the elementary matrix corresponding to the inverse elementary row operation.

Proof

Consider $E$ , $E^{\prime}$ elementary matrices of the same size corresponding to inverse row operations. We have:

$$ E^{\prime} E= I$$

because:

$$ E^{\prime} \left( EI \right) = I$$

$\square$