Line 1: | Line 1: | ||

== Four Fundamental Subspaces == | == Four Fundamental Subspaces == | ||

A matrix $A$ has four subspaces: | A matrix $A$ has four subspaces: | ||

− | * [[Column Space]] $C(A)$ | + | * [[Column Space]] $C(A)$ or $\text{ran}(A)$: Range of $A$ |

− | * [[Nullspace]] $N(A)$ | + | * [[Nullspace]] $N(A)$ or $\text{null}(A)$ |

* [[Row Space]] $C(A^T)$ of $A$ is the same as Column Space of $A^T$ | * [[Row Space]] $C(A^T)$ of $A$ is the same as Column Space of $A^T$ | ||

* Nullspace of $A^T$ (also called "Left Nullspace") | * Nullspace of $A^T$ (also called "Left Nullspace") | ||

Line 19: | Line 19: | ||

+ | == The Four Spaces == | ||

=== [[Column Space]] === | === [[Column Space]] === | ||

* $\text{dim } C(A) = r$, there are $r$ pivot columns | * $\text{dim } C(A) = r$, there are $r$ pivot columns | ||

* basis: columns of $A$ | * basis: columns of $A$ | ||

+ | * also called "range" | ||

+ | * $\text{ran}(A) = \{ \mathbf y \in \mathbb R^m \ : \ \mathbf y = A \mathbf x \ \forall \mathbf x \in \mathbb R^n \}$ | ||

+ | * if we think about [[Linear Transformation]] $T_{A}$ formed by $A$, this is what it does to an $n$-dimensional vector $\mathbf x$: it brings it to an $m$-dinesional vector $\mathbf y$ | ||

+ | * all such vectors $\mathbf y$ form the column space of $A$ | ||

Line 33: | Line 38: | ||

* $\text{dim } N(A) = n - r$ - the number of free variables | * $\text{dim } N(A) = n - r$ - the number of free variables | ||

* basis: special solutions for [[Homogeneous Systems of Linear Equations|$A\mathbf x = \mathbf 0$]] | * basis: special solutions for [[Homogeneous Systems of Linear Equations|$A\mathbf x = \mathbf 0$]] | ||

+ | * $N(A) = \text{null}(A) = \{ \mathbf x \in \mathbb R^n \ : \ A \mathbf x = \mathbf 0 \}$ | ||

Line 43: | Line 49: | ||

We know how to find the basis for all the subspaces | We know how to find the basis for all the subspaces | ||

* e.g. from using [[Gaussian Elimination]] transform the matrix to the echelon form and find them | * e.g. from using [[Gaussian Elimination]] transform the matrix to the echelon form and find them | ||

− | * but these bases are not "perfect". We want to use [[Orthogonal Vectors]] instead | + | * but these bases are not "perfect". We want to use [[Space Orthogonality|Orthogonal Vectors]] instead |

Line 64: | Line 70: | ||

* The Four Fundamental Subspaces: 4 Lines, G. Strang, [http://web.mit.edu/18.06/www/Essays/newpaper_ver3.pdf] | * The Four Fundamental Subspaces: 4 Lines, G. Strang, [http://web.mit.edu/18.06/www/Essays/newpaper_ver3.pdf] | ||

* [[Seminar Hot Topics in Information Management IMSEM (TUB)]] | * [[Seminar Hot Topics in Information Management IMSEM (TUB)]] | ||

+ | * [[Matrix Computations (book)]] | ||

[[Category:Linear Algebra]] | [[Category:Linear Algebra]] |

A matrix $A$ has four subspaces:

- Column Space $C(A)$ or $\text{ran}(A)$: Range of $A$
- Nullspace $N(A)$ or $\text{null}(A)$
- Row Space $C(A^T)$ of $A$ is the same as Column Space of $A^T$
- Nullspace of $A^T$ (also called "Left Nullspace")

Suppose we have an $m \times n$ matrix of rank $r$

- Nullspace of $A$ is orthogonal to the row space: $N(A) \; \bot \; C(A^T)$
- Left nullspace of $A$ is orthogonal to the column space: $N(A^T) \; \bot \; C(A)$
- see the proof in Space Orthogonality#Row space and Nullspace

- $\text{dim } C(A) = r$, there are $r$ pivot columns
- basis: columns of $A$
- also called "range"
- $\text{ran}(A) = \{ \mathbf y \in \mathbb R^m \ : \ \mathbf y = A \mathbf x \ \forall \mathbf x \in \mathbb R^n \}$
- if we think about Linear Transformation $T_{A}$ formed by $A$, this is what it does to an $n$-dimensional vector $\mathbf x$: it brings it to an $m$-dinesional vector $\mathbf y$
- all such vectors $\mathbf y$ form the column space of $A$

- $\text{dim } C(A^T) = r = \text{dim } C(A)$, there are $r$ pivot rows - the same dim as for Column Space
- Let $R$ be Row Reduced Echelon Form of $A$, then $C(A^T) = C(R^T)$
- basis: first $r$ rows of $R$

- $\text{dim } N(A) = n - r$ - the number of free variables
- basis: special solutions for $A\mathbf x = \mathbf 0$
- $N(A) = \text{null}(A) = \{ \mathbf x \in \mathbb R^n \ : \ A \mathbf x = \mathbf 0 \}$

- This is the nullspace of $A^T$ ($A^T$ is $n \times m$ matrix of rank $r$)
- $\text{dim } N(A^T) = m - r$ - there are $m$ columns, $m$ variables, and $m - r$ free variables

We know how to find the basis for all the subspaces

- e.g. from using Gaussian Elimination transform the matrix to the echelon form and find them
- but these bases are not "perfect". We want to use Orthogonal Vectors instead

SVD finds these bases:

- if $A V = U \Sigma$ then
- $\mathbf v_1, \ ... \ , \mathbf v_r$ is the basis for the row space $C(A^T)$
- $\mathbf v_{r+1}, \ ... \ , \mathbf v_{n}$ is the basis for the nullspace $N(A)$
- $\mathbf u_1, \ ... \ , \mathbf u_r$ is the basis for the column space $C(A)$
- $\mathbf u_{r+1}, \ ... \ , \mathbf u_{m}$ is the basis for the left nullspace $N(A^T)$

- Linear Algebra MIT 18.06 (OCW)
- The fundamental theorem of linear algebra, G. Strang [1]
- The Four Fundamental Subspaces: 4 Lines, G. Strang, [2]
- Seminar Hot Topics in Information Management IMSEM (TUB)
- Matrix Computations (book)