AdaBoost.SAMME

This variant, called Stagewise Additive Modeling using a Multi-class Exponential loss (SAMME), was proposed by Zhu, Rosset, Zou, and Hastie in Multi-class AdaBoost, Zhu J., Rosset S., Zou H., Hastie T., 01/2006. The goal is to adapt AdaBoost.M1 in order to work properly in multi-class scenarios. As this is a discrete version, its structure is almost the same, with a difference in the estimator weight computation. Let's consider a label dataset, Y:

Now, there are p different classes and it's necessary to consider that a random guess estimator cannot reach an accuracy equal to 0.5; therefore, the new estimator weights are computed as follows:

In this way, the threshold is pushed forward and α(t) will be zero when the following is true:

 

The following graph shows the plot of α(t) with p = 10:

Estimator weight plot as a function of the normalized weighted error sum when p = 10

Employing this correction, the boosting process can successfully cope with multi-class problems without the bias normally introduced by AdaBoost.M1 when p > 2 (α(t) > 0 when the error is less than an actual random guess, which is a function of the number of classes). As the performance of this algorithm is clearly superior, the majority of AdaBoost implementations aren't based on the original algorithm anymore (as already mentioned, for example, Scikit-Learn implements AdaBoost.SAMME and the real-valued version AdaBoost.SAMME.R). Of course, when p = 2, AdaBoost.SAMME is exactly equivalent to AdaBoost.M1.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.138.67.27