Another useful dimensionality reduction technique is called NMF. It again implements the same basic mathematical operations as PCA and ICA, but it has the additional constraint that it only operates on non-negative data. In other words, we cannot have negative values in our feature matrix if we want to use NMF; the resulting components of the decomposition will all have non-negative values as well.

In scikit-learn, NMF works exactly like ICA:

In [13]: nmf = decomposition.NMF()
In [14]: X2 = nmf.fit_transform(X)
In [15]: plt.plot(X2[:, 0], X2[:, 1], 'o')
...      plt.xlabel('first non-negative component')
...      plt.ylabel('second non-negative component')
...      plt.axis([-5, 20, -5, 10])
Out[15]: [-5, 20, -5, 10]

However, the resulting decomposition looks noticeably different from both PCA and ICA, which is a hallmark of NMF:

One of the things that we haven't covered, but that we would recommend you to research, is the idea of reducing dimensions by using only a subset of features rather than deriving new features. Also, how those features are found out is something you can learn more about online.

Now that we are familiar with some of the most common data decomposition tools, let's have a look at some common data types.