Vector Spaces, Linear Independence & Transformations

1. Introduction to Vector Spaces

1.1 Definition of a Vector Space

A vector space $V$ over a field $\mathbb{F}$ (typically $\mathbb{R}$ or $\mathbb{C}$) is a set equipped with two operations:

• Vector addition: $+:V\times V\to V$
• Scalar multiplication: $\cdot :\mathbb{F}\times V\to V$

Axioms (must hold for all $u,v,w\in V$ and $a,b\in \mathbb{F}$):

Example:

• ${\mathbb{R}}^n$ (all $n$-tuples of real numbers) is a vector space over $\mathbb{R}$.
• $P_n$ (all polynomials of degree $\le n$) is a vector space over $\mathbb{R}$.

---

1.2 Subspaces

A subspace $W$ of a vector space $V$ is a subset of $V$ that is itself a vector space under the operations of $V$.

Theorem: $W\subseteq V$ is a subspace if and only if:

1. $0\in W$ (non-empty),
2. Closed under addition: $u,v\in W⟹u+v\in W$,
3. Closed under scalar multiplication: $a\in \mathbb{F},u\in W⟹au\in W$.

Example (from past exams): Let $H=\left\{\left[\begin{array}{c}s\\ t\\ 0\end{array}\right]:s,t\in \mathbb{R}\right\}$. Show $H$ is a subspace of ${\mathbb{R}}^3$. Proof:

1. $0=\left[\begin{array}{c}0\\ 0\\ 0\end{array}\right]\in H$.
2. Let $u=\left[\begin{array}{c}{s}_1\\ t_1\\ 0\end{array}\right],v=\left[\begin{array}{c}{s}_2\\ t_2\\ 0\end{array}\right]\in H$. Then $u+v=\left[\begin{array}{c}{s}_1+s_2\\ t_1+t_2\\ 0\end{array}\right]\in H$.
3. Let $a\in \mathbb{R}$. Then $au=\left[\begin{array}{c}a{s}_1\\ a{t}_1\\ 0\end{array}\right]\in H$. ⇒ $H$ is a subspace.

Common Subspaces:

---

2. Linear Independence and Span

2.1 Linear Independence

A set of vectors $\{v_1,v_2,\dots ,v_k\}$ in $V$ is linearly independent if the only solution to: $$c_1{v}_1+c_2{v}_2+\cdots +c_k{v}_k=0$$ is $c_1=c_2=\cdots =c_k=0$. Otherwise, they are linearly dependent.

Example (from past exams): Are $v_1=\left[\begin{array}{c}1\\ 4\\ 0\end{array}\right],v_2=\left[\begin{array}{c}10\\ 2\\ 1\end{array}\right],v_3=\left[\begin{array}{c}-5\\ 0\\ 6\end{array}\right]$ linearly independent? Solution: Form the matrix $[v_1{v}_2{v}_3]$ and compute its determinant: $$\text{det}=1\cdot (2\cdot 6-1\cdot 0)-4\cdot (10\cdot 6-1\cdot (-5))+0\cdot (10\cdot 0-2\cdot (-5))=12-260=-248\ne 0.$$ Since the determinant is non-zero, the vectors are linearly independent.

Alternative Method (Row Reduction): Convert $[v_1{v}_2{v}_3]$ to reduced row echelon form (RREF). If the RREF has a pivot in every column, the vectors are independent.

---

2.2 Span and Basis

• Span: The set of all linear combinations of vectors $\{v_1,\dots ,v_k\}$ is their span, denoted $\text{span}\{v_1,\dots ,v_k\}$.
• Basis: A set $B=\{v_1,\dots ,v_n\}$ is a basis for $V$ if:
  1. $B$ is linearly independent,
  2. $\text{span}(B)=V$.

Example: Show that $\{(1,4,3),(0,3,1),(3,-5,4)\}$ is a basis for ${\mathbb{R}}^3$. Solution:

1. Linear Independence: Form the matrix and check determinant (non-zero ⇒ independent).
2. Span: Since ${\mathbb{R}}^3$ has dimension 3 and we have 3 independent vectors, they span ${\mathbb{R}}^3$. ⇒ Basis.

Dimension: The number of vectors in a basis for $V$, denoted $\dim (V)$.

---

3. Linear Transformations

3.1 Definition

A function $T:V\to W$ between vector spaces is a linear transformation if for all $u,v\in V$ and $a\in \mathbb{F}$:

1. $T(u+v)=T(u)+T(v)$ (additivity),
2. $T(au)=aT(u)$ (homogeneity).

Example (from past exams): Is $T(x,y)=(3x+y,5x+7y,x+3y)$ linear? Solution: Check the two conditions:

1. $T((x_1,y_1)+(x_2,y_2))=T(x_1+x_2,y_1+y_2)=(3(x_1+x_2)+(y_1+y_2),\dots )=T(x_1,y_1)+T(x_2,y_2)$.
2. $T(a(x,y))=T(ax,ay)=(3ax+ay,\dots )=aT(x,y)$. ⇒ Linear.

Non-Example: $T(x,y)=(x^2+y,x+y^2)$ is not linear (fails additivity: $T(1,0)+T(0,1)\ne T(1,1)$).

---

3.2 Matrix Representation of Linear Transformations

Every linear transformation $T:{\mathbb{R}}^n\to {\mathbb{R}}^m$ can be represented by an $m\times n$ matrix $A$ such that: $$T(\mathbf{x})=A\mathbf{x}.$$ Example: Let $A=\left[\begin{array}{cc}1& -3\\ 3& 5\\ -1& 7\end{array}\right]$, $v=\left[\begin{array}{c}2\\ -1\end{array}\right]$. Find $T(v)$. Solution: $$T(v)=Av=\left[\begin{array}{c}1\cdot 2+(-3)\cdot (-1)\\ 3\cdot 2+5\cdot (-1)\\ -1\cdot 2+7\cdot (-1)\end{array}\right]=\left[\begin{array}{c}5\\ 1\\ -9\end{array}\right].$$

---

3.3 Kernel and Range

• Kernel (Null Space): $\text{Null}(T)=\{x\in V:T(x)=0\}$.
• Range (Image): $\text{Range}(T)=\{T(x):x\in V\}$.

Theorem: For $T:V\to W$, $\dim (V)=\dim (\text{Null}(T))+\dim (\text{Range}(T))$ (Rank-Nullity Theorem).

---

4. Homogeneous and Non-Homogeneous Systems

4.1 Homogeneous Systems

A system $A\mathbf{x}=0$ is homogeneous. It always has at least the trivial solution $\mathbf{x}=0$.

Nontrivial Solutions Exist If: The system has infinitely many solutions, which occurs when $\det (A)=0$ (i.e., $A$ is singular).

Example (from past exams): Solve $x-2y+3z=0$, $-x+2y-4z=0$. Solution: The second equation is a multiple of the first ($-1\times \text{Equation 1}$). Thus, the system reduces to: $$x-2y+3z=0.$$ Express $x$ in terms of $y$ and $z$: $$x=2y-3z.$$ The general solution is: $$\mathbf{x}=\left[\begin{array}{c}2y-3z\\ y\\ z\end{array}\right]=y\left[\begin{array}{c}2\\ 1\\ 0\end{array}\right]+z\left[\begin{array}{c}-3\\ 0\\ 1\end{array}\right],y,z\in \mathbb{R}.$$

---

4.2 Non-Homogeneous Systems

A system $A\mathbf{x}=\mathbf{b}$ is non-homogeneous. It is:

• Consistent if it has at least one solution,
• Inconsistent if no solution exists (e.g., $0=1$).

Example (from past exams): Is $-2x_1-3x_2+4x_3=5$, $x_2-2x_3=4$, $x_1+3x_2-x_3=2$ consistent? Solution: Convert to augmented matrix and row reduce: $$\left[\begin{array}{ccccc}-2& -3& 4& |& 5\\ 0& 1& -2& |& 4\\ 1& 3& -1& |& 2\end{array}\right]$$ After row operations (swap R1 and R3, then eliminate): $$\left[\begin{array}{ccccc}1& 3& -1& |& 2\\ 0& 1& -2& |& 4\\ 0& 0& 1& |& 1\end{array}\right].$$ Back-substitution gives a unique solution $(x_1,x_2,x_3)=(5,6,1)$. ⇒ Consistent.

---

5. Augmented Matrices and Gaussian Elimination

5.1 Augmented Matrix

For a system $A\mathbf{x}=\mathbf{b}$, the augmented matrix is $[A|\mathbf{b}]$.

Example (from past exams): Find the general solution for: $$\left[\begin{array}{ccccc}1& 6& 2& |& -5\\ -1& 0& 3& |& 1\\ 0& -1& -2& |& 3\end{array}\right].$$ Solution:

1. Row Reduction (Gaussian Elimination):
   • $R_2\leftarrow R_2+R_1$: $$\left[\begin{array}{ccccc}1& 6& 2& |& -5\\ 0& 6& 5& |& -4\\ 0& -1& -2& |& 3\end{array}\right].$$
   • $R_3\leftarrow 6R_3+R_2$: $$\left[\begin{array}{ccccc}1& 6& 2& |& -5\\ 0& 6& 5& |& -4\\ 0& 0& -7& |& 22\end{array}\right].$$
2. Back-Substitution:
   • From $R_3$: $-7z=22⟹z=-\frac{22}{7}$.
   • From $R_2$: $6y+5z=-4⟹y=\frac{-4-5z}{6}$.
   • From $R_1$: $x+6y+2z=-5⟹x=-5-6y-2z$.
   • Substitute $z$ and solve for $y$ and $x$.

General Solution: The system has a unique solution (no free variables in RREF). $$x=\frac{37}{7},y=-\frac{19}{7},z=-\frac{22}{7}.$$

---

6. Key Comparisons

---

7. Applications

1. Computer Graphics: Linear transformations (rotation, scaling) are used to manipulate 3D models.
2. Machine Learning: Vector spaces represent data points; linear independence ensures features are unique.
3. Engineering: Solving circuit equations or structural analysis using matrix methods.
4. Economics: Input-output models in economics rely on linear systems.

---

8. Exam Tips

What Examiners Look For:

1. Definitions: Always state definitions precisely (e.g., "A vector space $V$ over $\mathbb{F}$ is a set...").
2. Proofs for Subspaces: Show closure under addition, scalar multiplication, and non-emptiness.
3. Linear Independence: Use either:
   • Determinant method (for square matrices),
   • Row reduction to RREF and check for free variables,
   • Direct substitution (for small systems).
4. Linear Transformations: Verify both additivity and homogeneity.
5. Homogeneous Systems: Recall that nontrivial solutions exist only if the determinant is zero.
6. Augmented Matrices: Always perform row operations correctly and interpret the RREF properly (unique solution, no solution, or free variables).
7. Basis Questions: To show a set is a basis, prove both linear independence and spanning.

Common Mistakes to Avoid:

• Assuming all vectors are independent: Always check!
• Ignoring the zero vector: $\{0\}$ is linearly dependent.
• Incorrect row operations: Forgetting to perform the same operation on all rows.
• Misapplying definitions: E.g., confusing span and linear independence.
• Arithmetic errors: Double-check calculations, especially in determinants.

Quick Revision Checklist:

---

Practice Problems (Solve These!)

1. Show that $W=\{(x,y,z)\in {\mathbb{R}}^3:x+y+z=0\}$ is a subspace of ${\mathbb{R}}^3$.
2. Determine if $\{(1,2,3),(0,1,1),(1,3,4)\}$ is linearly independent.
3. Let $T:{\mathbb{R}}^2\to {\mathbb{R}}^2$ be defined by $T(x,y)=(x+y,2x-y)$. Show $T$ is linear and find its matrix representation.
4. Solve the homogeneous system: $$\{\begin{array}{l}x+2y-z=0\\ 2x-y+3z=0\\ x+y+z=0\end{array}.$$
5. Is the transformation $T(x,y)=(x^2+y,x-y^2)$ linear? Justify.

---

Final Note

Mastering this unit is critical for the rest of linear algebra. Focus on:

• Understanding definitions (they form the backbone of proofs).
• Practicing row reduction (essential for solving systems).
• Visualizing linear independence (geometrically in ${\mathbb{R}}^2$ and ${\mathbb{R}}^3$).

Good luck with your exams! 🚀