Classification algorithms study how to automatically learn to make accurate predictions based on observations. Starting from a set of predefined class labels, the algorithm gives each piece of data input a class label in accordance with the training model. If there are just two distinction classes, we talk about binary classification; otherwise, we go for multi-class classification. In more detail, each category corresponds to a different label; the algorithm attaches a label to each instance, which simply indicates which class the data belongs to. A procedure that can perform this function is commonly called a classifier. The following graph shows an automated classification of the Iris flower based on petal length and height, using discriminant analysis:

Classification has some analogy with regression: as well as regression, classification uses known labels of a training dataset to predict the response of the new dataset. The main difference between regression and classification is that regression is used to predict continuous values, whereas classification works with discrete data.

For example, regression can be used to predict the future price of oil based on its prices over the last 10 years. However, we should use the classification method to predict whether the price of oil will rise or decrease in the near future. In the first case, we use continuous data as a prediction and choose a continuous data response (the precise price of oil). In the second case, starting with continuous values (the price of oil over the last 10 years), we begin by classifying the various phases where a growth/diminution of price has been recorded; then we use that classification to predict a relative trend in the near future.

To approach a classification problem, several methods are available:

• Decision trees
• Naive Bayes algorithms
• Discriminant analysis
• K-nearest neighbors

Such algorithms provide appreciable performance when applied to problems with well-defined limits, where inputs follow a specific set of attributes and where the output has discrete values. The classification process takes advantage of the experience developed by evaluating new inputs through matching with previously observed and correctly labeled models.

A standard classification process includes the following steps:

1. Training the algorithm: In this step, the machine learning begins to work with the definition of the model and the next training. The model starts to extract knowledge from large amounts of data that we had available (training dataset); for this data, the couplings (instance or class) are available.
2. Testing the algorithm: In this step, we use the information learned in the previous step to see whether the model actually works. The evaluation of an algorithm is for seeing how well the model approximates the real system. In the case of supervised learning, we have some known values that we can use to evaluate the algorithm. If we are not satisfied, we can return to the previous steps, change some things, and retry the test.

Each classification algorithm, among those listed above, has its own peculiarities and is based on certain assumptions. We cannot say a priori which algorithm will provide us with the best performance. This is because the choice of the algorithm appropriate for our activity requires practice. In fact it is clear that no classifier will work best in all possible scenarios.

So, how will we choose the one that best fits our needs? To do this, simply compare the performance of different learning algorithms to select the best model for the specific problem; these may differ in the number of features or samples, the amount of noise in a dataset, and whether the classes are separable linearly or not. It will be enough to take the necessary patience and apply different algorithms before choosing the one that works best for our application.