In this section, we will discuss the DT algorithm in detail. A comparative analysis of Naive Bayes and DT will be discussed too. DTs are commonly considered as a supervised learning technique used for solving classification and regression tasks. A DT is simply a decision support tool that uses a tree-like graph (or a model of decisions) and their possible consequences, including chance event outcomes, resource costs, and utility. More technically, each branch in a DT represents a possible decision, occurrence, or reaction in terms of statistical probability.

Compared to Naive Bayes, DT is a far more robust classification technique. The reason is that at first DT splits the features into training and test set. Then it produces a good generalization to infer the predicted labels or classes. Most interestingly, DT algorithm can handle both binary and multiclass classification problems.

For instance, in the preceding example figure, DTs learn from the admission data to approximate a sine curve with a set of if...else decision rules. The dataset contains the record of each student who applied for admission, say to an American university. Each record contains the graduate record exam score, CGPA score, and the rank of the column. Now we will have to predict who is competent based on these three features (variables). DTs can be used to solve this kind of problem after training the DT model and pruning unwanted branches of the tree. In general, a deeper tree signifies more complex decision rules and a better fitted model. Therefore, the deeper the tree, the more complex the decision rules and the more fitted the model.