

Line 17: 
Line 17: 
 We can look differently at the results of [[Eigendecomposition]] of $A$   We can look differently at the results of [[Eigendecomposition]] of $A$ 
   
−  * $A = Q \Lambda Q^T = \begin{bmatrix}  +  * <math>A = Q \Lambda Q^T = \begin{bmatrix} 
  & &  \\    & &  \\ 
 \mathbf q_1 & \cdots & \mathbf q_n \\   \mathbf q_1 & \cdots & \mathbf q_n \\ 
Line 31: 
Line 31: 
 & \vdots & \\   & \vdots & \\ 
  & \mathbf q_n^T &  \\    & \mathbf q_n^T &  \\ 
−  \end{bmatrix}$  +  \end{bmatrix}</math> 
 * can represent it as $A = Q \Lambda Q^T = \sum \lambda_i \mathbf q_i \mathbf q_i^T$  sum of [[Outer Product]]s   * can represent it as $A = Q \Lambda Q^T = \sum \lambda_i \mathbf q_i \mathbf q_i^T$  sum of [[Outer Product]]s 
 * each of these outer products can be seen as a [[Projection MatricesProjection Matrix]]   * each of these outer products can be seen as a [[Projection MatricesProjection Matrix]] 
Revision as of 00:26, 14 November 2015
Spectral Theorem
Spectral Theorem is also sometimes called Principal Axis Theorem
Theorem:
 Every Symmetric Matrix can be factorized as $A = Q \Lambda Q^T$
 with real eigenvalues $\Lambda$ and orthonormal eigenvectors in the columns of $Q$
The factorization is Eigendecomposition
 Spectral Theorem is a special case for symmetric matrices
 See the proof in the Symmetric Matrices article
We can look differently at the results of Eigendecomposition of $A$
 [math]A = Q \Lambda Q^T = \begin{bmatrix}
 & &  \\
\mathbf q_1 & \cdots & \mathbf q_n \\
 & &  \\
\end{bmatrix}
\begin{bmatrix}
\lambda_1 & & \\
& \ddots & \\
& & \lambda_n \\
\end{bmatrix}
\begin{bmatrix}
 & \mathbf q_1^T &  \\
& \vdots & \\
 & \mathbf q_n^T &  \\
\end{bmatrix}[/math]
 can represent it as $A = Q \Lambda Q^T = \sum \lambda_i \mathbf q_i \mathbf q_i^T$  sum of Outer Products
 each of these outer products can be seen as a Projection Matrix
 a projection matrix is $P_i = \cfrac{\mathbf q_i \mathbf q_i^T}{\ \mathbf q_i \^2} = \mathbf q_i \mathbf q_i^T$
 so symmetric matrix can be represented as a combination of mutually orthogonal projection matrices
Applications
Principal Component Analysis
 The Spectral Theorem guarantees that we will find an orthogonal basis in PCA
 Because the Covariance Matrix $C = \cfrac{1}{n  1} X^T X$ is symmetric and PositiveDefinite
Sources