Line 1: Line 1:
 
== Four Fundamental Subspaces ==
 
== Four Fundamental Subspaces ==
 
A matrix $A$ has four subspaces:  
 
A matrix $A$ has four subspaces:  
* [[Column Space]] $C(A)$
+
* [[Column Space]] $C(A)$ or $\text{ran}(A)$: Range of $A$
* [[Nullspace]] $N(A)$
+
* [[Nullspace]] $N(A)$ or $\text{null}(A)$
 
* [[Row Space]] $C(A^T)$ of $A$ is the same as Column Space of $A^T$
 
* [[Row Space]] $C(A^T)$ of $A$ is the same as Column Space of $A^T$
 
* Nullspace of $A^T$ (also called "Left Nullspace")
 
* Nullspace of $A^T$ (also called "Left Nullspace")
Line 19: Line 19:
  
  
 +
== The Four Spaces ==
 
=== [[Column Space]] ===
 
=== [[Column Space]] ===
 
* $\text{dim } C(A) = r$, there are $r$ pivot columns
 
* $\text{dim } C(A) = r$, there are $r$ pivot columns
 
* basis: columns of $A$  
 
* basis: columns of $A$  
 +
* also called "range"
 +
* $\text{ran}(A) = \{ \mathbf y \in \mathbb R^m \ : \ \mathbf y = A \mathbf x \ \forall \mathbf x \in \mathbb R^n  \}$
 +
* if we think about [[Linear Transformation]] $T_{A}$ formed by $A$, this is what it does to an $n$-dimensional vector $\mathbf x$: it brings it to an $m$-dinesional vector $\mathbf y$
 +
* all such vectors $\mathbf y$ form the column space of $A$
  
  
Line 33: Line 38:
 
* $\text{dim } N(A) = n - r$ - the number of free variables
 
* $\text{dim } N(A) = n - r$ - the number of free variables
 
* basis: special solutions for [[Homogeneous Systems of Linear Equations|$A\mathbf x = \mathbf 0$]]
 
* basis: special solutions for [[Homogeneous Systems of Linear Equations|$A\mathbf x = \mathbf 0$]]
 +
* $N(A) = \text{null}(A) = \{ \mathbf x \in \mathbb R^n \ : \ A \mathbf x = \mathbf 0 \}$
  
  
Line 43: Line 49:
 
We know how to find the basis for all the subspaces
 
We know how to find the basis for all the subspaces
 
* e.g. from using [[Gaussian Elimination]] transform the matrix to the echelon form and find them
 
* e.g. from using [[Gaussian Elimination]] transform the matrix to the echelon form and find them
* but these bases are not "perfect". We want to use [[Orthogonal Vectors]] instead
+
* but these bases are not "perfect". We want to use [[Space Orthogonality|Orthogonal Vectors]] instead
  
  
Line 64: Line 70:
 
* The Four Fundamental Subspaces: 4 Lines, G. Strang, [http://web.mit.edu/18.06/www/Essays/newpaper_ver3.pdf]
 
* The Four Fundamental Subspaces: 4 Lines, G. Strang, [http://web.mit.edu/18.06/www/Essays/newpaper_ver3.pdf]
 
* [[Seminar Hot Topics in Information Management IMSEM (TUB)]]
 
* [[Seminar Hot Topics in Information Management IMSEM (TUB)]]
 +
* [[Matrix Computations (book)]]
  
 
[[Category:Linear Algebra]]
 
[[Category:Linear Algebra]]

Latest revision as of 13:43, 27 June 2017

Four Fundamental Subspaces

A matrix $A$ has four subspaces:

  • Column Space $C(A)$ or $\text{ran}(A)$: Range of $A$
  • Nullspace $N(A)$ or $\text{null}(A)$
  • Row Space $C(A^T)$ of $A$ is the same as Column Space of $A^T$
  • Nullspace of $A^T$ (also called "Left Nullspace")



Some Properties

Suppose we have an $m \times n$ matrix of rank $r$

Orthogonality

  • Nullspace of $A$ is orthogonal to the row space: $N(A) \; \bot \; C(A^T)$
  • Left nullspace of $A$ is orthogonal to the column space: $N(A^T) \; \bot \; C(A)$
  • see the proof in Space Orthogonality#Row space and Nullspace


The Four Spaces

Column Space

  • $\text{dim } C(A) = r$, there are $r$ pivot columns
  • basis: columns of $A$
  • also called "range"
  • $\text{ran}(A) = \{ \mathbf y \in \mathbb R^m \ : \ \mathbf y = A \mathbf x \ \forall \mathbf x \in \mathbb R^n \}$
  • if we think about Linear Transformation $T_{A}$ formed by $A$, this is what it does to an $n$-dimensional vector $\mathbf x$: it brings it to an $m$-dinesional vector $\mathbf y$
  • all such vectors $\mathbf y$ form the column space of $A$


Row Space

  • $\text{dim } C(A^T) = r = \text{dim } C(A)$, there are $r$ pivot rows - the same dim as for Column Space
  • Let $R$ be Row Reduced Echelon Form of $A$, then $C(A^T) = C(R^T)$
  • basis: first $r$ rows of $R$


Nullspace

  • $\text{dim } N(A) = n - r$ - the number of free variables
  • basis: special solutions for $A\mathbf x = \mathbf 0$
  • $N(A) = \text{null}(A) = \{ \mathbf x \in \mathbb R^n \ : \ A \mathbf x = \mathbf 0 \}$


Left Nullspace

  • This is the nullspace of $A^T$ ($A^T$ is $n \times m$ matrix of rank $r$)
  • $\text{dim } N(A^T) = m - r$ - there are $m$ columns, $m$ variables, and $m - r$ free variables


Singular Value Decomposition

We know how to find the basis for all the subspaces


SVD finds these bases:

  • if $A V = U \Sigma$ then
  • $\mathbf v_1, \ ... \ , \mathbf v_r$ is the basis for the row space $C(A^T)$
  • $\mathbf v_{r+1}, \ ... \ , \mathbf v_{n}$ is the basis for the nullspace $N(A)$
  • $\mathbf u_1, \ ... \ , \mathbf u_r$ is the basis for the column space $C(A)$
  • $\mathbf u_{r+1}, \ ... \ , \mathbf u_{m}$ is the basis for the left nullspace $N(A^T)$




Sources