Whitening

Whitening takes data whose variables are correlated and turns it into uncorrelated data, all on the same scale. It comes before almost any serious analysis: without it, an algorithm mistakes the shape of the data for signal.

The plot above shows what happens. On the left the original data, stretched along a diagonal because the two variables move together. Then the centred data. Then the two transforms, PCA and ZCA: the cloud becomes round, the correlation is gone, and ZCA hands it back in the original orientation while PCA rotates it onto the principal axes.

I implemented the main variants — via Cholesky decomposition, via eigenvalues, plus an alternative based on singular value decomposition — and compared them on real series of accelerometer readings, electricity load, exchange rates and light sensors, not only on generated data.