Line 4: Line 4:
  
  
=== $C(A)$ ===
+
=== The Column Space $C(A)$ ===
 
Let $A$ be $m \times n$ matrix:
 
Let $A$ be $m \times n$ matrix:
 
* $A = \left[ \mathop{a_1}\limits_|^| \, \mathop{a_2}\limits_|^| \ \cdots \  \mathop{a_n}\limits_|^| \right]$
 
* $A = \left[ \mathop{a_1}\limits_|^| \, \mathop{a_2}\limits_|^| \ \cdots \  \mathop{a_n}\limits_|^| \right]$
Line 11: Line 11:
 
* $C(A)$ is in $\mathbb R^r$ space where $r \leqslant n$ is the $A$'s [[Rank (Matrix)|Rank]]  
 
* $C(A)$ is in $\mathbb R^r$ space where $r \leqslant n$ is the $A$'s [[Rank (Matrix)|Rank]]  
 
* so the dimensionality is at most the number of columns, and at least the rank of the matrix  
 
* so the dimensionality is at most the number of columns, and at least the rank of the matrix  
 +
 +
 +
<img width="50%" src="http://alexeygrigorev.com/projects/imsem-ws14-lina/img-svg/diagram1.svg" />
 +
 +
 +
It's also called the ''range'' of $A$:
 +
* $\text{ran}(A) = \{ \mathbf y \in \mathbb R^m \ : \ \mathbf y = A \mathbf x \ \forall \mathbf x \in \mathbb R^n  \}$
 +
* if we think about [[Linear Transformation]] $T_{A}$ formed by $A$, this is what it does to an $n$-dimensional vector $\mathbf x$: it brings it to an $m$-dinesional vector $\mathbf y$
 +
* all such vectors $\mathbf y$ form the column space of $A$
 +
  
  
Line 61: Line 71:
 
For example, the following can solve it:
 
For example, the following can solve it:
 
* $\mathbf 0_4$, because $\mathbf x = \mathbf 0_3$ will solve it
 
* $\mathbf 0_4$, because $\mathbf x = \mathbf 0_3$ will solve it
* $\begin{bmatrix}
+
* <math>\begin{bmatrix}
 
1 \\
 
1 \\
 
2 \\
 
2 \\
 
3 \\
 
3 \\
 
4 \\
 
4 \\
\end{bmatrix}$ - one of the columns, so $\mathbf x = \begin{bmatrix}
+
\end{bmatrix}</math> - one of the columns, so <math>\mathbf x = \begin{bmatrix}
 
1 \\
 
1 \\
 
0 \\
 
0 \\
 
0 \\
 
0 \\
\end{bmatrix}$
+
\end{bmatrix}</math>
  
  
Line 86: Line 96:
 
== Sources ==
 
== Sources ==
 
* [[Linear Algebra MIT 18.06 (OCW)]]
 
* [[Linear Algebra MIT 18.06 (OCW)]]
 +
* [[Matrix Computations (book)]]
  
 
[[Category:Linear Algebra]]
 
[[Category:Linear Algebra]]

Latest revision as of 13:48, 27 June 2017

Column Space

A column space $C(A)$ of a matrix $A$ is a subspace formed by columns of $A$


The Column Space $C(A)$

Let $A$ be $m \times n$ matrix:

  • $A = \left[ \mathop{a_1}\limits_|^| \, \mathop{a_2}\limits_|^| \ \cdots \ \mathop{a_n}\limits_|^| \right]$
  • the columns of $A$ form a subspace - a hyperplane through the origin
  • 57705070eeef45ee91d525a65ba2ea75.png
  • $C(A)$ is in $\mathbb R^r$ space where $r \leqslant n$ is the $A$'s Rank
  • so the dimensionality is at most the number of columns, and at least the rank of the matrix



It's also called the range of $A$:

  • $\text{ran}(A) = \{ \mathbf y \in \mathbb R^m \ : \ \mathbf y = A \mathbf x \ \forall \mathbf x \in \mathbb R^n \}$
  • if we think about Linear Transformation $T_{A}$ formed by $A$, this is what it does to an $n$-dimensional vector $\mathbf x$: it brings it to an $m$-dinesional vector $\mathbf y$
  • all such vectors $\mathbf y$ form the column space of $A$


It's a subspace:

  • if we take any vectors from $C(A)$, the linear combination will still be $C(A)$ (by definition)
  • f51d03568a64453d91ca9e198318de93.png


Example

Suppose we have a matrix $A \in \mathbb R^{3 \times 2}$

$A = \begin{bmatrix} 1 & 3 \\ 2 & 3 \\ 4 & 1 \\ \end{bmatrix}$

Subspace from columns - $C(A)$ - the Column Space of $A$:

  • we cannot just take the two columns and call it a subspace:
  • it also must include all linear combinations of these columns
  • these linear combinations of two vectors form a plane - a subspace $\mathbb R^2$ in the space $\mathbb R^3$
  • since we include all possible combinations, we're guaranteed to have a subspace
  • cf5432f561ec4f14888e8b376c5f438b.png
  • $v_1$ and $v_2$ are 1st and 2nd columns of $A$ - they form a plane through the origin


System $A \mathbf x = \mathbf b$

Column Space $C(A)$ of $A$ is important: the system $A \mathbf x = \mathbf b$ has the solution only when $\mathbf b \in C(A)$

For example:

$A = \begin{bmatrix} 1 & 1 & 2 \\ 2 & 1 & 3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \\ \end{bmatrix}$.

  • There are 3 columns and they are 4-dim vectors
  • so $C(A)$ is a subspace $\mathbb R^4$
  • but how big it is? is it the entire $\mathbb R^4$? No - we have only 3 vectors, so it's at most $\mathbb R^3$


Since there are only 3 columns, in $A \mathbf x = \mathbf b$

  • $\mathbf x \in \mathbb R^3$ and $\mathbf b \in \mathbb R^4$
  • does it always have a solution? no: we have 4 equations and 3 unknowns
  • there are many $\mathbf b$'s that can't solve the system
  • but there are some that can: they are linear combinations of the columns - so those $\mathbf b$ that are from $C(A)$


For example, the following can solve it:

  • $\mathbf 0_4$, because $\mathbf x = \mathbf 0_3$ will solve it
  • [math]\begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \\ \end{bmatrix}[/math] - one of the columns, so [math]\mathbf x = \begin{bmatrix} 1 \\ 0 \\ 0 \\ \end{bmatrix}[/math]


if $\mathbf b \not \in C(A)$ there's no way to solve the system


What's the dimension of $C(A)$?

  • $\text{dim } C(A) = r$ where $r$ -rank of $A$
  • the easiest way to determine it - is calculate the number of pivot columns during Gaussian Elimination
  • in this case, $\text{dim } C(A) = 2$ because the rank is 2 (the 3rd column is a linear combination of 1st and 2nd)



Sources

Machine Learning Bookcamp: Learn machine learning by doing projects. Get 40% off with code "grigorevpc".

Share your opinion