Score and evaluate the model

As we anticipated in the previous section, so far we have used 70% of the data for model training; now it is possible to use the remaining 30% to test its functioning. To do this, we will use the Score Model module. This module scores predictions for a trained classification or regression model. Follow this procedure:

  1. Select the Score Model module. The Score Model module is located at the following path of the module palette: Machine Learning | Score. Drag the Score Model module to the experiment canvas area.
  2. Connect the output port of the Train Model module to the left input port of the Score Model module.
  3. Connect the test data output port of the Split Data module (right output port) to the right input port of the Score Model.
  4. Run the experiment.

In the following screenshot, the essential elements used in the previous procedure are highlighted:

To analyze the test simulation results, just right-click on the output port of Score Model and select Visualize. The output shows the estimated values for breast cancer and the values known from the test data, as shown in the following screenshot:


From the analysis of the screenshot, it is possible to notice that the forecast seems to have provided good results. But let's try to evaluate the results obtained more rigorously. To evaluate the quality of the results, we can use the Evaluate Model module. This module evaluates the results of a classification or regression model with standard metrics. Follow this procedure now:

  1. Select the Evaluate Model module. The Evaluate Model module is located at the following path of the module palette: Machine Learning | Evaluate. Drag the Score Model module into the experiment canvas area.
  2. Connect the output port of the Score Model module to the left input port of the Evaluate Model module.
  3. Run the experiment.

To view the Evaluate Model output, right-click on the output port and select Visualize. The following metrics are reported:

In the preceding screenshot, the terms are defined as follows:

  • True Positive (TP): All cases where the predicted and actual values are both true (1 = malignant).
  • True Negative (TN): All cases where the predicted value is false and the actual value is also false (0 = benign).
  • False Positive (FP): This is a case where we predict something as positive (true), but it is actually negative.
  • False Negative (FN): Where we predict something as false, but actually it is true.
  • Accuracy: This is the measure of how good our model is. It is expected to be closer to 1 if our model is performing well.
  • Precision and Recall: These are again ratios between the TP with (TP + FP) and TP with (TP + FN) respectively. These ratios determine how relevant our predictions are compared to the actual values.
  • F1 Score: This is computed as the weighted average of precision and recall between 0 and 1, where the ideal F1 score value is 1.
  • AUC: This measures the area under the curve plotted with true positives on the y axis and false positives on the x axis.

It should be noted that our model is able to adapt very well to the data. In fact, the evaluation of the model provided an accuracy value equal to 0.98.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.191.240.222