In this chapter, we learned about a new class of models capable of capturing a non-linear relationship, in contrast to the classical linear models we had explored so far. We saw how decision trees learn rules to partition the feature space into regions that yield predictions and thus segment the input data into specific regions.

Decision trees are very useful because they provide unique insights into the relationships between features and target variables, and we saw how to visualize the sequence of decision rules encoded in the tree structure.

Unfortunately, a decision tree is prone to overfitting. We learned that ensemble models and the bootstrap aggregation method manages to overcome some of the shortcomings of decision trees and render them useful, as components of much more powerful composite models. 

In the next chapter, we will explore another ensemble model, which has come to be considered one of the most important machine learning algorithms.