Eigenvalues and Eigenvectors – Key Concepts and Applications

Introduction

In linear algebra a matrix $A\in {\mathbb{R}}^{n\times n}$ acts on a vector $x$ producing another vector $Ax$.
If $x$ is only stretched or shrunk but not rotated, it is called an eigenvector and the stretching factor is the eigenvalue.
Mathematically,
$$Ax=\lambda x,\lambda \in \mathbb{C},x\ne 0.$$
The set of all eigenvalues of $A$ is called the spectrum of $A$.
Eigenpairs are fundamental because they diagonalise matrices, simplify differential equations, and reveal intrinsic properties of systems.

Definitions

The characteristic polynomial $p(\lambda )=\det (A-\lambda I)$ is a degree‑$n$ polynomial whose roots are the eigenvalues.

The graph shows the cubic characteristic polynomial of a 3×3 matrix with eigenvalues 1, 2, 3.

How to Compute Eigenvalues

1. Direct Method (Small Matrices)

For $2\times 2$ or $3\times 3$ matrices, compute $\det (A-\lambda I)=0$ symbolically.

Worked Example (2×2)
Find eigenvalues and eigenvectors of
$$A=\left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right].$$

1. Characteristic polynomial:
   $$\det \left[\begin{array}{cc}2-\lambda & 1\\ 1& 2-\lambda \end{array}\right]=(2-\lambda {)}^2-1=0$$ $${\lambda }^2-4\lambda +3=0$$ $$(\lambda -1)(\lambda -3)=0$$ Hence ${\lambda }_1=1,{\lambda }_2=3$.

2. Eigenvectors:
   For $\lambda =1$: solve $(A-I)x=0$
   $$\left[\begin{array}{cc}1& 1\\ 1& 1\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=0\Rightarrow x_1=-x_2.$$ Choose $x_2=1$, then $x=\left[\begin{array}{c}1\\ -1\end{array}\right]$.

For $\lambda =3$: solve $(A-3I)x=0$
$$\left[\begin{array}{cc}-1& 1\\ 1& -1\end{array}\right]\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right]=0\Rightarrow x_1=x_2.$$ Choose $x_2=1$, then $x=\left[\begin{array}{c}1\\ 1\end{array}\right]$.

Result
$${\lambda }_1=1,v_1=\left[\begin{array}{c}1\\ -1\end{array}\right],{\lambda }_2=3,v_2=\left[\begin{array}{c}1\\ 1\end{array}\right].$$

The two eigenvectors are orthogonal and lie along the lines $y=x$ and $y=-x$.

2. Power Iteration (Large Sparse Matrices)

Power iteration finds the dominant eigenpair $({\lambda }_{\max },v_{\max })$.

Algorithm

1. Choose random vector $b^{(0)}$.
2. For $k=1,2,\dots$:
   $$b^{(k)}=\frac{A{b}^{(k-1)}}{\Vert A{b}^{(k-1)}\Vert }.$$
3. Approximate eigenvalue: ${\lambda }^{(k)}=\frac{(b^{(k)}{)}^{T}A{b}^{(k)}}{(b^{(k)}{)}^T{b}^{(k)}}$.

Trace Example (3×3)
Let
$$A=\left[\begin{array}{ccc}4& 1& 0\\ 1& 3& 1\\ 0& 1& 2\end{array}\right].$$
Start with $b^{(0)}=\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right]$.

After a few iterations, ${\lambda }_{\max }\approx 5.20$ and $v_{\max }\approx \left[\begin{array}{c}0.743\\ 0.577\\ 0.332\end{array}\right]$.

flowchart TD
  A["Start with random \(b^{(0)}\)"] --> B["Compute \(b^{(k)}=A\,b^{(k-1)}\)"];
  B --> C["Normalize \(b^{(k)}\)"];
  C --> D["Estimate \(\lambda^{(k)}\)"];
  D --> E["Check convergence"];
  E -->|"Yes"| F["Return \((\lambda_{\max},v_{\max})\)"];
  E -->|"No"| B;

3. QR Algorithm (Full Spectrum)

The QR algorithm iteratively decomposes $A_k=Q_k{R}_k$ and sets $A_{k+1}=R_k{Q}_k$.
After enough iterations, $A_k$ converges to an upper triangular matrix whose diagonal entries are the eigenvalues.

Key Steps

1. $A_0=A$.
2. For $k=0,1,\dots$:
   • Compute QR factorisation $A_k=Q_k{R}_k$.
   • Set $A_{k+1}=R_k{Q}_k$.

The algorithm is robust and works for any square matrix.

sequenceDiagram
  participant A as \(A_k\)
  participant Q as \(Q_k\)
  participant R as \(R_k\)
  A->>Q: QR decomposition
  A->>R: QR decomposition
  Q->>A: Multiply \(R_kQ_k\)
  R->>A: Multiply \(R_kQ_k\)
  Note over A: \(A_{k+1}=R_kQ_k\)

Inverse Iteration and Deflation

Inverse iteration refines eigenpairs near a chosen shift $\mu$.
Deflation removes already found eigenvalues to compute the remaining ones.

Inverse Iteration
$$b^{(k)}=\frac{(A-\mu I{)}^{-1}{b}^{(k-1)}}{\Vert (A-\mu I{)}^{-1}{b}^{(k-1)}\Vert }.$$

Deflation
After finding ${\lambda }_1$ and $v_1$, form
$$A^{\prime }=A-{\lambda }_1{v}_1{v}_1^T$$
and apply the algorithm again to $A^{\prime }$.

Eigen-Decomposition and Applications

1. Diagonalisation

If $A$ is diagonalizable,
$$A=V\Lambda V^{-1},$$
where $V$ contains eigenvectors and $\Lambda$ is diagonal with eigenvalues.
This simplifies matrix powers: $A^k=V{\Lambda }^k{V}^{-1}$.

2. Stability Analysis

For a linear system $\dot{x}=Ax$, the sign of the real parts of eigenvalues determines stability.

• All $\Re (\lambda )<0$: stable.
• Any $\Re (\lambda )>0$: unstable.

3. Vibrations

In mechanical systems, eigenvalues of the stiffness‑mass matrix give natural frequencies.

4. Data Science

Principal Component Analysis (PCA) uses eigenvectors of the covariance matrix to find directions of maximum variance.

Comparison of Algorithms

In the real world

1. Google PageRank – The web link matrix $G$ is stochastic. Its dominant eigenvector gives page importance.
   Product: Google Search.
   Idea used: Power iteration on $G$ to compute PageRank scores.

2. Ncell MIMO Channel Capacity – The channel matrix $H$ has eigenvalues that determine achievable data rates via the water‑filling algorithm.
   Product: 4G/5G base stations.
   Idea used: Eigen-decomposition of $H^{H}H$ to allocate power across spatial streams.

3. NEPSE Price Prediction – The correlation matrix of stock returns is eigen‑decomposed to identify market factors.
   Product: Stock market analytics.
   Idea used: PCA on correlation matrix to reduce dimensionality before forecasting.

Advantages and Disadvantages

Exam tip

• Key formulas: $p(\lambda )=\det (A-\lambda I)$, power iteration update, QR step.
• Common questions:
  1. Compute eigenvalues of a given matrix.
  2. Explain the QR algorithm and its convergence.
  3. Apply power iteration to a sparse matrix and interpret results.
• Practice: Work through at least one 3×3 example by hand and one 5×5 example using QR.
• Time management: Allocate 10 min for characteristic polynomial, 15 min for algorithm explanation, 15 min for worked example.