|
|
Line 17: |
Line 17: |
| We can look differently at the results of [[Eigendecomposition]] of $A$ | | We can look differently at the results of [[Eigendecomposition]] of $A$ |
| | | |
− | * $A = Q \Lambda Q^T = \begin{bmatrix} | + | * <math>A = Q \Lambda Q^T = \begin{bmatrix} |
| | & & | \\ | | | & & | \\ |
| \mathbf q_1 & \cdots & \mathbf q_n \\ | | \mathbf q_1 & \cdots & \mathbf q_n \\ |
Line 31: |
Line 31: |
| & \vdots & \\ | | & \vdots & \\ |
| - & \mathbf q_n^T & - \\ | | - & \mathbf q_n^T & - \\ |
− | \end{bmatrix}$ | + | \end{bmatrix}</math> |
| * can represent it as $A = Q \Lambda Q^T = \sum \lambda_i \mathbf q_i \mathbf q_i^T$ - sum of [[Outer Product]]s | | * can represent it as $A = Q \Lambda Q^T = \sum \lambda_i \mathbf q_i \mathbf q_i^T$ - sum of [[Outer Product]]s |
| * each of these outer products can be seen as a [[Projection Matrices|Projection Matrix]] | | * each of these outer products can be seen as a [[Projection Matrices|Projection Matrix]] |
Revision as of 00:26, 14 November 2015
Spectral Theorem
Spectral Theorem is also sometimes called Principal Axis Theorem
Theorem:
- Every Symmetric Matrix can be factorized as $A = Q \Lambda Q^T$
- with real eigenvalues $\Lambda$ and orthonormal eigenvectors in the columns of $Q$
The factorization is Eigendecomposition
- Spectral Theorem is a special case for symmetric matrices
- See the proof in the Symmetric Matrices article
We can look differently at the results of Eigendecomposition of $A$
- [math]A = Q \Lambda Q^T = \begin{bmatrix}
| & & | \\
\mathbf q_1 & \cdots & \mathbf q_n \\
| & & | \\
\end{bmatrix}
\begin{bmatrix}
\lambda_1 & & \\
& \ddots & \\
& & \lambda_n \\
\end{bmatrix}
\begin{bmatrix}
- & \mathbf q_1^T & - \\
& \vdots & \\
- & \mathbf q_n^T & - \\
\end{bmatrix}[/math]
- can represent it as $A = Q \Lambda Q^T = \sum \lambda_i \mathbf q_i \mathbf q_i^T$ - sum of Outer Products
- each of these outer products can be seen as a Projection Matrix
- a projection matrix is $P_i = \cfrac{\mathbf q_i \mathbf q_i^T}{\| \mathbf q_i \|^2} = \mathbf q_i \mathbf q_i^T$
- so symmetric matrix can be represented as a combination of mutually orthogonal projection matrices
Applications
Principal Component Analysis
- The Spectral Theorem guarantees that we will find an orthogonal basis in PCA
- Because the Covariance Matrix $C = \cfrac{1}{n - 1} X^T X$ is symmetric and Positive-Definite
Sources