Principal Component Analysis is the most popular and commonly used technique for Dimensionality Reduction
Suppose we want to reduce from 2D to 1D
We want to find a line which would give us the smallest square distance from the data points to their projection
Before running PCA it's a good idea to perform Feature Scaling
To reduce from $N$-dim to $K$-dim