This method is based on a theorem that states that a matrix X d x N can be decomposed as follows:
Here:
- U is a d x d unitary matrix
- ∑ is a d x N diagonal matrix where the diagonal entries si are called singular values
- V is an N x N unitary matrix
In our case, X can be composed by the feature's vectors 
, where each 
is a column. We can reduce the number of dimensions of each feature vector d, approximating the singular value decomposition. In practice, we consider only the largest singular values 
so that:
t represents the dimension of the new reduced space where the feature vectors are projected. A vector x(i) is transformed in the new space using the following formula:
This means that the matrix 
(not 
) represents the feature vectors in the t dimensional space.
Note that it is possible to show that this method is very similar to the PCA; in fact, the scikit-learn library uses SVD to implement PCA.