In order to run our app, we will need to execute the main function routine (in chapter7.py) that loads the pre-trained cascade classifier and the pre-trained multi-layer perceptron, and applies them to each frame of the webcam live stream.

However, this time, instead of collecting more training samples, we will select the radio button that says Test. This will trigger an EVT_RADIOBUTTON event, which binds to FaceLayout._on_testing, disabling all training-related buttons in the GUI and switching the app to the testing mode. In this mode, the pre-trained MLP classifier is applied to every frame of the live stream, trying to predict the current facial expression.

As promised earlier, we now return to FaceLayout._process_frame:

def _process_frame(self, frame):
    """ detects face, predicts face label in testing mode """

Unchanged from what we discussed earlier, the method begins by detecting faces in a downscaled grayscale version of the current frame:

    success, frame, self.head = self.faces.detect(frame)

However, in the testing mode, there is more to the function:

    # in testing mode: predict label of facial expression
    if success and self.testing.GetValue():

In order to apply our pre-trained MLP classifier to the current frame, we need to apply the same preprocessing to the current frame as we did with the entire training set. After aligning the head region, we apply PCA by using the pre-loaded basis vectors (self.pca_V) and mean values (self.pca_m):

    # if face found: preprocess (align)
    success, head = self.faces.align_head(self.head)
    if success:
        # extract features using PCA (loaded from file)
        X, _, _ = homebrew.extract_features([head.flatten()], self.pca_V, self.pca_m)

Then, we are ready to predict the class label of the current frame:

    # predict label with pre-trained MLP
    label = self.MLP.predict(np.array(X))[0]

Since the predict method already returns a string label, all that is left to do is to display it above the bounding box in the current frame:

    # draw label above bounding box
    cv2.putText(frame, str(label), (x,y-20), cv2.FONT_HERSHEY_COMPLEX, 1, (0,255,0), 2)
    break # need only look at first, largest face

Finally, we are done!

    return frame

The end result looks like the following:

Although the classifier has only been trained on (roughly) 100 training samples, it reliably detects my various facial expressions in every frame of the live stream, no matter how distorted my face seemed to be at the moment. This is a good indication that the neural network that was learned neither underfits nor overfits the data, since it is capable of predicting the correct class labels even for new data samples.