We have previously said that RBM is often used as a feature in a wide variety of classification problems. It's time to see how to do it. The first thing to do is to use the BernoulliRBM function of the sklearn.neural_network module.

In the sklearn library, the sklearn.neural_network module includes models based on neural networks. In this module, the BernoulliRBM function fits a Bernoulli RBM. An RBM with binary visible units and binary hidden units is returned. The parameters are estimated using Stochastic Maximum Likelihood (SML), also known as Persistent Contrastive Divergence (PCD). First, we will set the architecture of the model:

RbmModel = BernoulliRBM(random_state=0, verbose=True)

Then, we will fit the model with the training data:

FitRbmModel = RbmModel.fit_transform(X_train, Y_train)

The fit_transform method fits the transformer to X_train and Y_train with optional parameter fit_params, and returns a transformed version of X_train. In this case, no optional parameters are used.

If you remember, our purpose is to use the Rbm model to extract the features that will then be used by the logistic regression model to classify the data. So, the first part has already been performed—we already have the features extracted in the FitRbmModel variable. The time has come to create the logistic regression model. To do this, we will use LogisticRegression function of the sklearn.linear_model module, as follows:

LogModel = linear_model.LogisticRegression()

We now set coefficients of the features in the decision function equal to the features extracted from the rbm model:

LogModel.coef_ = FitRbmModel

Now we can build the classifier. To do this, we will use the Pipeline function of the sklearn.pipeline module:

Classifier = Pipeline(steps=[('RbmModel', RbmModel), ('LogModel', LogModel)])

The purpose of the pipeline is to assemble several steps that can be cross-validated together while setting different parameters. For this, it enables setting parameters of the various steps using their names, as in the previous code. A step's estimator may be replaced entirely by setting the parameter with its name to another estimator, or a transformer removed by setting to None. The classifier is now ready; we just have to train it:

LogModel.fit(X_train, Y_train)
Classifier.fit(X_train, Y_train)

First, the logistic regression model is trained and then the classifier. We just have to make predictions. Recall that for doing this, we have an unused dataset available: X_test and Y_test. To check the performance of the classifier, we will compare the forecasts with the real data:

print ("The RBM model:")
print ("Predict: ", Classifier.predict(X_test))
print ("Real:    ", Y_test)

The following screenshot shows the results returned:

Finally, to better understand the model performance, we will calculate the confusion matrix. In a confusion matrix, our classification results are compared to real data. The strength of a confusion matrix is that it identifies the nature of the classification errors as well as their quantities. In this matrix, the diagonal cells show the number of cases that were correctly classified; all the other cells show the misclassified cases. To calculate the confusion matrix, we can use the ConfusionMatrix() function contained in pandas library as follows:

CM = ConfusionMatrix(Y_test, Classifier.predict(X_test))
CM.print_stats()

In the following code, the results returned by the ConfusionMatrix() function are shown:

population: 114
P: 72
N: 42
PositiveTest: 87
NegativeTest: 27
TP: 71
TN: 26
FP: 16
FN: 1
TPR: 0.9861111111111112
TNR: 0.6190476190476191
PPV: 0.8160919540229885
NPV: 0.9629629629629629
FPR: 0.38095238095238093
FDR: 0.1839080459770115
FNR: 0.013888888888888888
ACC: 0.8508771929824561
F1_score: 0.8930817610062893
MCC: 0.6866235389841608
informedness: 0.6051587301587302
markedness: 0.7790549169859515
prevalence: 0.631578947368421
LRP: 2.588541666666667
LRN: 0.022435897435897433
DOR: 115.37500000000003
FOR: 0.037037037037037035

Several bits of information are returned; in particular, we can notice that the accuracy of the model is 0.85.