We have loaded our data and selected the best features, and we are ready to learn some classification models. Let's begin with basic decision trees.

In Weka, a decision tree is implemented within the J48 class, which is a reimplementation of Quinlan's famous C4.5 decision tree learner (Quinlan, 1993).

We will make a decision tree by using the following steps:

1. We initialize a new J48 decision tree learner. We can pass additional parameters with a string table—for instance, the tree pruning that controls the model complexity (refer to Chapter 1, Applied Machine Learning Quick Start). In our case, we will build an un-pruned tree; hence, we will pass a single -U parameter, as follows:

J48 tree = new J48(); 
String[] options = new String[1]; 
options[0] = "-U"; 
 
tree.setOptions(options); 

2. We will call the buildClassifier(Instances) method to initialize the learning process:

tree.buildClassifier(data); 

3. The built model is now stored in a tree object. We can provide the entire J48 unpruned tree by calling the toString() method:

System.out.println(tree); 

The output will be as follows:

    J48 unpruned tree
    ------------------
    
    feathers = false
    |   milk = false
    |   |   backbone = false
    |   |   |   airborne = false
    |   |   |   |   predator = false
    |   |   |   |   |   legs <= 2: invertebrate (2.0)
    |   |   |   |   |   legs > 2: insect (2.0)
    |   |   |   |   predator = true: invertebrate (8.0)
    |   |   |   airborne = true: insect (6.0)
    |   |   backbone = true
    |   |   |   fins = false
    |   |   |   |   tail = false: amphibian (3.0)
    |   |   |   |   tail = true: reptile (6.0/1.0)
    |   |   |   fins = true: fish (13.0)
    |   milk = true: mammal (41.0)
    feathers = true: bird (20.0)
    
    Number of Leaves  : 9
    
    Size of the tree : 17
  

The tree in the output has 17 nodes in total and 9 of them are terminal (Leaves).

Another way to present the tree is to leverage the built-in TreeVisualizer tree viewer, as follows:

TreeVisualizer tv = new TreeVisualizer(null, tree.graph(), new PlaceNode2()); 
JFrame frame = new javax.swing.JFrame("Tree Visualizer"); 
frame.setSize(800, 500); 
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); 
frame.getContentPane().add(tv); 
frame.setVisible(true); 
tv.fitToScreen(); 

The preceding code results in the following output frame:

The decision process starts at the top node, also known as the root node. The node label specifies the attribute value that will be checked. In our example, first, we check the value of the feathers attribute. If the feather is present, we follow the right-hand branch, which leads us to the leaf labeled bird, indicating that there are 20 examples supporting this outcome. If the feather is not present, we follow the left-hand branch, which leads us to the milk attribute. We check the value of the attribute again, and we follow the branch that matches the attribute value. We repeat the process until we reach a leaf node.

We can build other classifiers by following the same steps: initialize a classifier, pass the parameters controlling the model complexity, and call the buildClassifier(Instances) method.

In the next section, you will learn how to use a trained model to assign a class label to a new example whose label is unknown.