Although the RBF kernel makes for a good default kernel, it is not always the one that works best for our problem. The only real way to know which kernel works best on our data is to try them all and compare the classification performance across models. There are strategic ways to perform this so-called hyperparameter tuning, which we'll talk about in detail in Chapter 11, Selecting the Right Model with Hyperparameter Tuning.

What if we don't know how to do hyperparameter tuning properly yet?

Well, I'm sure you remember the first step in data understanding, visualize the data. Visualizing the data could help us understand if a linear SVM was powerful enough to classify the data, in which case there would be no need for a more complex model. After all, we know that if we can draw a straight line to separate the data, a linear classifier is all we need. However, for a more complex problem, we would have to think harder about what shape the optimal decision boundary should have.

In more general terms, we should think about the geometry of our problem. Some questions we might ask ourselves are as follows:

• Does visualizing the dataset make it obvious which kind of decision rule would work best? For example, is a straight line good enough (in which case, we would use a linear SVM)? Can we group different data points into blobs or hotspots (in which case, we would use an RBF kernel)?
• Is there a family of data transformations that do not fundamentally change our problem? For example, could we flip the plot on its head, or rotate it, and get the same result? Our kernel should reflect that.
• Is there a way to preprocess our data that we haven't tried yet? Spending some time on feature engineering can make the classification problem much easier in the end. Perhaps we might even be able to use a linear SVM on the transformed data.

As a follow-up exercise, think about which kernel we would use to classify the following datasets:

Think you know the answer? Code it up!

Before we wrap up this chapter, let's learn about multi-class classification in SVMs. So, without any further ado, let's dive into it!