Quasiorthonormal Encoding

Our example with the Japanese car makes:

Make	One-Hot	Quasiorthonornal Code
Toyota	(1,0,0,0,0)	(0.851, 0.000, -0.526)
Honda	(0,1,0,0,0)	(0.526, -0.851, 0.000)
Subaru	(0,0,1,0,0)	(0.000, -0.526, 0.851)
Nissan	(0,0,0,1,0)	(0.851, 0.000, 0.526)
Mitsubishi	(0,0,0,0,1)	(-0.526, -0.851, 0.000)

If you think of one-hot encoding as encoding categories with unit vectors in $N$ dimensions ($\mathbb{R}^5$ in this case), then you are encoding your categories onto an orthonormal basis (the set of unit vectors in $\mathbb{R}^N$. Quasiorthonormal encoding relaxes the orthonormal requirement and substitute a quasiorthonormal "basis". The above table shows 5 quasiorthonormal vectors in 3 dimensions having a minimum mutual angle of aboout $63^\circ$.

There is substantial mathematical theory on what is quasiorthogonality and quasiorthonormality, how you construct it, how to decode it including a counterpart to the softmax function. All of which is presented in much greater detail in the accompanying proceedings paper.

Back to Classic Encodings