Anyone who works with data analysis and machine learning will come to understand that no method is ideal or universal. This is why there are so many methods. Researchers and enthusiasts have been searching for years for a compromise between the accuracy, simplicity, and interpretability of various models. Moreover, how can we increase the accuracy of the model, preferably without changing its essence? One way to improve the accuracy of models is to create and train model ensembles—that is, sets of models used to solve the same problem. The ensemble training methodology is the training of a final set of simple classifiers, with the subsequent merging of the results of their predictions into a single forecast of the aggregated algorithm.

This chapter describes what ensemble learning is, what types of ensembles exist, and how they can help to obtain better predictive performance. In this chapter, we will also implement examples of these approaches with different C++ libraries.

The following topics will be covered in this chapter:

• An overview of ensemble learning
• Learning about decision trees and random forests
• Examples of using C++ libraries for creating ensembles