Eigenvalues and Eigenvectors

Most vectors change direction when a matrix acts on them. Eigenvectors are the special non-zero vectors whose direction is preserved (or exactly reversed) for a nonzero real eigenvalue, or which are collapsed to zero for eigenvalue 0; complex eigenvectors are understood through the algebraic equation below. The matrix scales an eigenvector, by a factor called the eigenvalue. Eigenvalues summarise a matrix compactly: their sum is the trace, their product is the determinant, and they decide invertibility, powers, stability and diagonalisability. Many eigenvalue facts follow from properties alone, without solving a cubic.

Let $A$ be $n \times n$. A scalar $\lambda$ is an eigenvalue of $A$ if there is a non-zero vector $v$ with

Such $v$ is an eigenvector for $\lambda$. The zero vector is never an eigenvector, but $\lambda = 0$ can be an eigenvalue. Rewriting as $(A - \lambda I)v = 0$, a non-zero solution exists if and only if $A - \lambda I$ is singular:

$p(\lambda) = \det(A - \lambda I)$ is a polynomial of degree $n$ (some books use $\det(\lambda I - A)$, which differs by the factor $(-1)^n$ and has the same roots). Over $\mathbb{C}$ it has exactly $n$ roots counted with multiplicity; a real matrix can have complex eigenvalues, which come in conjugate pairs. The eigenspace of $\lambda$ is $N(A - \lambda I)$: all eigenvectors for $\lambda$ together with $0$.

If the eigenvalues (with algebraic multiplicity) are $\lambda_1, \ldots, \lambda_n$, then

For $2 \times 2$ $A$: $\lambda^2 - (\operatorname{tr} A)\lambda + \det A = 0$. Consequently, $A$ is singular if and only if 0 is an eigenvalue.

Always $1 \le \text{GM} \le \text{AM}$.

If $Av = \lambda v$, then with the same eigenvector $v$:

Also: $A^T$ has the same eigenvalues as $A$ (same characteristic polynomial) but generally different eigenvectors; $AB$ and $BA$ have the same eigenvalues for square $A, B$; similar matrices $B = P^{-1}AP$ share the characteristic polynomial, hence eigenvalues, trace, determinant and rank. Eigenvalues of $A + B$ are not sums of eigenvalues of $A$ and $B$ in general.

Triangular and diagonal matrices: the eigenvalues are the diagonal entries.

$A$ is diagonalisable if $A = PDP^{-1}$ with $D$ diagonal; the columns of $P$ are eigenvectors and the diagonal of $D$ holds the matching eigenvalues.

Diagonalisation makes powers easy: $A^k = PD^kP^{-1}$.

Every square matrix satisfies its own characteristic equation: if $p(\lambda) = \det(\lambda I - A)$, then $p(A) = 0$. For $2 \times 2$: $A^2 - (\operatorname{tr} A)A + (\det A)I = 0$. Uses: express $A^{-1}$ or high powers of $A$ as a polynomial of degree less than $n$ in $A$.

The geometric multiplicity of the eigenvalue 0 is $n - \operatorname{rank} A$. Since GM $\le$ AM, $\operatorname{rank} A \ge$ the number of non-zero eigenvalues counted with algebraic multiplicity, with equality when $A$ is diagonalisable. Equality can fail otherwise: The nilpotent $\begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}$ has rank 1 but no non-zero eigenvalue.

A real symmetric $A$ is positive definite if $x^TAx > 0$ for every $x \ne 0$. The following are equivalent: $A$ is positive definite; all eigenvalues of $A$ are positive; all leading principal minors of $A$ are positive (Sylvester's criterion); $A = LL^T$ with $L$ lower triangular with positive diagonal (Cholesky, see LU Decomposition). Positive semidefinite means $x^TAx \ge 0$, equivalently all eigenvalues $\ge 0$; for this weaker property the leading minors are not enough: $\begin{pmatrix} 0 & 0 \\ 0 & -1 \end{pmatrix}$ has leading minors 0 and 0 but eigenvalue $-1$.

$A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$: trace 7, determinant 10, so $\lambda^2 - 7\lambda + 10 = 0$, $\lambda = 5, 2$. For $\lambda = 5$: $A - 5I = \begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix}$, eigenvector $(1, 1)$. For $\lambda = 2$: $A - 2I = \begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix}$, eigenvector $(1, -2)$. With $P = \begin{pmatrix} 1 & 1 \\ 1 & -2 \end{pmatrix}$, $A^n = P\operatorname{diag}(5^n, 2^n)P^{-1}$; for instance, the $(1, 1)$ entry of $A^n$ is $(2 \cdot 5^n + 2^n)/3$.

$A = \begin{pmatrix} 2 & 1 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 3 \end{pmatrix}$. Eigenvalues $2, 2, 3$. $A - 2I$ has rank 2, so GM(2) = 1 $<$ AM(2) = 2: not diagonalisable.

A $3 \times 3$ matrix has eigenvalues $1, 2, 3$. Then $\operatorname{tr} A = 6$, $\det A = 6$; $A^{-1}$ has $1, 1/2, 1/3$; $A^2 - 2A + I = (A - I)^2$ has $0, 1, 4$ and is singular; $\operatorname{adj} A$ has $6, 3, 2$ and determinant $36 = 6^2$. $A$ is diagonalisable (distinct eigenvalues).

$A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$: $A^2 - 5A - 2I = 0$. Multiply by $A^{-1}$: $A^{-1} = \frac{1}{2}(A - 5I) = \begin{pmatrix} -2 & 1 \\ 3/2 & -1/2 \end{pmatrix}$.

$\begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$ has eigenvalues 3 and 1 with orthogonal eigenvectors $(1, 1)$ and $(1, -1)$. The rotation by $90^\circ$, $\begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$, has $\lambda^2 + 1 = 0$, so $\lambda = \pm i$: no real eigenvector, as expected since every direction turns.

$J_n$ (every entry 1) has rank 1, so 0 has GM $n - 1$; the trace $n$ must be the remaining eigenvalue, with eigenvector $(1, \ldots, 1)$. Eigenvalues: $n$ once, $0$ with multiplicity $n - 1$. Hence $J_n + cI$ has eigenvalues $n + c$ and $c$ ($n - 1$ times), and $\det(J_n + cI) = (n + c)c^{n-1}$.

$A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$: trace 7, determinant 10, so $\lambda = 5, 2$, with eigenvectors $(1, 1)^T$ and $(1, -2)^T$. With $P = \begin{pmatrix} 1 & 1 \\ 1 & -2 \end{pmatrix}$ and $P^{-1} = \frac13\begin{pmatrix} 2 & 1 \\ 1 & -1 \end{pmatrix}$,

For $k = 1$ this returns $A$; for $k = 3$ it gives $\begin{pmatrix} 86 & 39 \\ 78 & 47 \end{pmatrix}$. For large $k$, $A^k \approx 5^k \cdot \frac13\begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix}$: the largest eigenvalue dominates.

$T = \begin{pmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{pmatrix}$ has leading principal minors $2, 3, 4$, all positive, so $T$ is positive definite. Its eigenvalues are $2 - \sqrt2$, $2$, $2 + \sqrt2$, all positive; their sum is $6 = \operatorname{tr}T$ and their product is $4 = \det T$. The sum of their squares is $16 = \operatorname{tr}(T^2)$, not $(\operatorname{tr}T)^2 = 36$.