The trained model needs to be thoroughly validated before using it for AI applications. With AI applications, the idea is to complement human intelligence and so it becomes even more important to ensure that the model is validated on heterogeneous data samples (evaluation datasets) before deploying it. The model needs to pass a high threshold not only on the predefined evaluation sample but on the new datasets that were never seen by the model. We can evaluate the model using two primary categories of validation: holdout and cross-validation.

Both approaches use a test set to assess the model output (that is, data that is not seen by the algorithm). Using the data that we used to develop the model to test it is not recommended. This is because our model will remember the entire training set, and thus will predict the correct label at any point in the training set. This is known as overfitting. Because of its speed, simplicity, and versatility, the holdout approach is useful. Nonetheless, this approach is often associated with high uncertainty, as variations in the training and test dataset can lead to significant differences in accuracy estimates.