I’ve been looking at versions of Adaboost that are less sensitive to noise such as Softboost. Softboost works by ignoring a number of outliers set by the user (the parameter $v$), finding weak learners that are not highly correlated with the weak learners already in the boosted learner mix and updating the distribution by KL projection onto the set of distributions restricted to those uncorrelated to the mistakes of the latest learner and not placing too much weight on any particular data point. Softboost avoids over fitting by stopping when no feasible point is found for the KL projection.

In “Soft Margins for AdaBoost”, Ratsch, Onoda, and Muller, generalize Adaboost by adding a softening parameter $\phi$ to the distribution update step. They relate soft boosting to simulated annealing and minimization of a generalized exponential loss function. The paper has numerous helpful graphs and experimental data.

Related Posts via Categories

• “Stacked Generalization”
• “Rapid object detection using a boosted cascade of simple features”
• Product of Experts
• “Rotation Forest: A New Classifier Ensemble Method”
• Using Bregman Distance to Grow Decision Trees
• How to use Expert Advice
• Cooperative Ensemble Learning System (CELS)
• AdaBoost.NC
• Adaboost and Bregman Distances
• Hierarchical Optimistic Optimization (HOO)