Our example with the Japanese car makes:
|Toyota||(1,0,0,0,0)||(0.851, 0.000, -0.526)|
|Honda||(0,1,0,0,0)||(0.526, -0.851, 0.000)|
|Subaru||(0,0,1,0,0)||(0.000, -0.526, 0.851)|
|Nissan||(0,0,0,1,0)||(0.851, 0.000, 0.526)|
|Mitsubishi||(0,0,0,0,1)||(-0.526, -0.851, 0.000)|
If you think of one-hot encoding as encoding categories with unit vectors in dimensions ( in this case), then you are encoding your categories onto an orthonormal basis (the set of unit vectors in . Quasiorthonormal encoding relaxes the orthonormal requirement and substitute a quasiorthonormal "basis". The above table shows 5 quasiorthonormal vectors in 3 dimensions having a minimum mutual angle of aboout .
There is substantial mathematical theory on what is quasiorthogonality and quasiorthonormality, how you construct it, how to decode it including a counterpart to the
softmax function. All of which is presented in much greater detail in the accompanying proceedings paper.
Back to Classic Encodings