Seminar on Computational Learning and Adaptation

 
DynaBoost: Combining Boosted Hypotheses in a Dynamic Way

Eddy Mayoraz
Motorola, Lexicus Division
Palo Alto, CA
eddym@lexicus.mot.com

Ensemble learning  techniques such as Bagging or Boosting provide an efficient  way of enhancing  the performances of  simple (weak) classifiers.  A large number of weak learners are usually combined in a very simple way. This work explores some possibilities of more elaborated recombinations of  the learners, seeking either an improvement of the final result or a saving on the number of weak learners.   In particular, a  dynamic combination  is investigated, where the weighting  factors associated to each weak learner are functions of the input.  The resulting algorithm thus falls between Boosting and an incremental mixture of experts model.  Empirical comparisons between AdaBoost and DynaBoost show that a dynamic combination significantly improves the results when weak  learners (e.g., perceptrons)  are  used, while  the difference  in performance is small when the learners are more powerful (e.g., MLPs).

Joint work with Perry Moerland.

Return to the seminar schedule