Bias is the propensity of an ML model to consistently learn the same thing. Consistency refers to the results of repeated iterations on variations on the same dataset. The higher the bias, the more off target the resulting learned model tends to be. If the bias is lower, the resulting trained models will consistently be more on target.

A high bias model will have a large error rate on both the training data and the test data. This is referred to as underfitting. In other words, a more complex model could fit the data better in both situations (training and testing).

The following example compares a linear model with low bias in regards to the dataset, and a high bias linear model in regards to a different dataset. We use the same simple model to show that the level of bias is not necessarily related to the choice of ML model. Although some models have a higher propensity to bias, it is the combination of the model and the data that matters: