Line 9: | Line 9: | ||
=== Orthogonal Matrix === | === Orthogonal Matrix === | ||
What about a matrix form? | What about a matrix form? | ||
− | * The second part of the definition: | + | * The second part of the definition: <math>\mathbf q_i^T \mathbf q_j = |
\begin{cases} | \begin{cases} | ||
1 & \text{if } i \ne j \\ | 1 & \text{if } i \ne j \\ | ||
0 & \text{if } i = j | 0 & \text{if } i = j | ||
− | \end{cases} | + | \end{cases}</math> |
* how do we put it in a matrix form? | * how do we put it in a matrix form? | ||
− | * Consider a matrix $Q$ whose columns are vectors | + | * Consider a matrix $Q$ whose columns are vectors \mathbf q_1, \ ... \ , \mathbf q_n$: |
− | * let | + | * let <math>Q = \Bigg[ \mathop{\mathbf q_1}\limits_|^| \ \mathop{\mathbf q_2}\limits_|^| \ \cdots \ \mathop{\mathbf q_n}\limits_|^| \Bigg]</math> |
− | * | + | * <math>Q^T Q = |
\begin{bmatrix} | \begin{bmatrix} | ||
- \ \mathbf q_1^T - \\ | - \ \mathbf q_1^T - \\ | ||
Line 29: | Line 29: | ||
& & \ddots & \\ | & & \ddots & \\ | ||
0 & 1 & \cdots & 1 | 0 & 1 & \cdots & 1 | ||
− | \end{bmatrix} | + | \end{bmatrix}</math> by our definition! |
* so $Q^T Q = I$ | * so $Q^T Q = I$ | ||
* such $Q$'s are called ''Orthogonal Matrices'' | * such $Q$'s are called ''Orthogonal Matrices'' | ||
Line 45: | Line 45: | ||
=== Identity Matrices === | === Identity Matrices === | ||
Identity matrices are orthogonal: | Identity matrices are orthogonal: | ||
− | * | + | * <math>Q = \begin{bmatrix} |
1 & 0 & 0 \\ | 1 & 0 & 0 \\ | ||
0 & 1 & 0 \\ | 0 & 1 & 0 \\ | ||
0 & 0 & 1 \\ | 0 & 0 & 1 \\ | ||
− | \end{bmatrix} = I | + | \end{bmatrix} = I</math> |
* $Q^T Q = I I = I$ | * $Q^T Q = I I = I$ | ||
Line 55: | Line 55: | ||
=== [[Permutation Matrices]] === | === [[Permutation Matrices]] === | ||
[[Permutation Matrices]] are orthogonal | [[Permutation Matrices]] are orthogonal | ||
− | * consider | + | * consider <math>Q = \begin{bmatrix} |
0 & 0 & 1 \\ | 0 & 0 & 1 \\ | ||
1 & 0 & 0 \\ | 1 & 0 & 0 \\ | ||
0 & 1 & 0 \\ | 0 & 1 & 0 \\ | ||
− | \end{bmatrix} | + | \end{bmatrix}</math> |
− | * then | + | * then <math>Q^T = \begin{bmatrix} |
0 & 1 & 0 \\ | 0 & 1 & 0 \\ | ||
0 & 0 & 1 \\ | 0 & 0 & 1 \\ | ||
1 & 0 & 0 \\ | 1 & 0 & 0 \\ | ||
− | \end{bmatrix} | + | \end{bmatrix}</math> and indeed $Q^T Q = I$ |
* also note that $Q^T$ is also orthogonal | * also note that $Q^T$ is also orthogonal | ||
Line 70: | Line 70: | ||
=== [[Rotation Matrices]] === | === [[Rotation Matrices]] === | ||
[[Rotation Matrices]] are also orthogonal | [[Rotation Matrices]] are also orthogonal | ||
− | * let | + | * let <math>Q = \begin{bmatrix} |
\cos \theta & -\sin \theta \\ | \cos \theta & -\sin \theta \\ | ||
\sin \theta & \cos \theta \\ | \sin \theta & \cos \theta \\ | ||
− | \end{bmatrix} | + | \end{bmatrix}</math> |
* it's orthogonal | * it's orthogonal | ||
+ | |||
+ | |||
+ | === [[Reflection Matrices]] === | ||
+ | They are also orthogonal (add example) | ||
=== Not Orthogonal Example === | === Not Orthogonal Example === | ||
Not orthogonal: | Not orthogonal: | ||
− | * | + | * <math>S = \begin{bmatrix} |
1 & 1 \\ | 1 & 1 \\ | ||
1 & -1 \\ | 1 & -1 \\ | ||
− | \end{bmatrix} | + | \end{bmatrix}</math> |
− | * why? | + | * why? <math>S^T S = \begin{bmatrix} |
1 & 1 \\ | 1 & 1 \\ | ||
1 & -1 \\ | 1 & -1 \\ | ||
Line 92: | Line 96: | ||
2 & 0 \\ | 2 & 0 \\ | ||
0 & 2 \\ | 0 & 2 \\ | ||
− | \end{bmatrix} | + | \end{bmatrix}</math> |
* how to fix it? they are not unit vectors, so need to normalize it: | * how to fix it? they are not unit vectors, so need to normalize it: | ||
− | * | + | * <math>Q = \cfrac{1}{\sqrt 2} \begin{bmatrix} |
1 & 1 \\ | 1 & 1 \\ | ||
1 & -1 \\ | 1 & -1 \\ | ||
− | \end{bmatrix} | + | \end{bmatrix}</math> |
− | * now | + | * now <math>Q^T Q = \cfrac{1}{2} \begin{bmatrix} |
2 & 0 \\ | 2 & 0 \\ | ||
0 & 2 \\ | 0 & 2 \\ | ||
− | \end{bmatrix} = I | + | \end{bmatrix} = I</math> |
* this one is orthogonal | * this one is orthogonal | ||
Vectors $\mathbf q_1, \ ... \ , \mathbf q_n$ are orthonormal if they are orthogonal and unit vectors
What about a matrix form?
A matrix $Q$ is orthogonal if
What's special about being square?
Identity matrices are orthogonal:
Permutation Matrices are orthogonal
Rotation Matrices are also orthogonal
They are also orthogonal (add example)
Not orthogonal:
Why is it good to have orthogonal matrices?
Orthogonal matrices are very nice because it's very easy to invert them
How do we make matrices orthogonal?
Also,
If $Q$ is orthogonal matrix, then $Q^T$ is orthogonal as well
If $Q_1$ and $Q_2$ are orthogonal, so is $Q_1 \cdot Q_2$
$Q$ preserves the $L_2$ norm:
$Q$ preserves the angle between $\mathbf x$ and $\mathbf y$