In this chapter, we reviewed two classification techniques: KNN and SVM. The goal was to discover how these techniques work and ascertain the differences between them, by building and comparing models on a common dataset. KNN involved both unweighted and weighted nearest neighbor algorithms, and for SVM, only a linear model was developed, which outperformed all other models.

We examined how to use Recursive Feature Elimination to find an optimal set of features for both methods. We used the extremely versatile caret package to train the models. We expanded our exploration of model performance using a confusion matrix, and the relevant statistics that one can derive from the matrix. We'll now use tree-based classifiers, which are very powerful and very popular.