In overfitting, your model fails to generalize. You will determine the overfitting model when it performs well on the training set but poorly on the test set. This typically indicates that the model is too flexible for the amount of training data, and this flexibility allows it to memorize the data, including noise. Overfitting corresponds to high variance, where small changes in the training data result in big changes to the results.

In underfitting, your model fails to capture essential patterns in the training dataset. Typically, underfitting indicates the model is too simple or has too few explanatory variables. An underfit model is not flexible enough to model real patterns and corresponds to high bias, which indicates the results show a systematic lack of fit in a certain region.

The following graph illustrates the clear difference between overfitting versus underfitting as they correspond to a model with good fit:

In the preceding graphs, the ML model is trying to categorize between two data point categories illustrated by the red points and green crosses. The ML model is trying to determine whether a customer will buy a given product or not. The graph shows predictions from three different ML models. You can see an overfitted model (on the right) traversing through all red data points in training and failing to generalize the algorithm for real-world data outside of the training dataset. On the other hand, the underfitted model (on the left) leaves out several data points and produces an inaccurate result. A good model (shown in the middle) provides clear predictions of the data points in most of the cases. Creating a good ML model is like creating art and you can find the right fit with model tuning.

The ML algorithm is at the heart of the overall ML workflow, which is categorized into supervised and unsupervised learning.