No. 1005

Title:

Boosting method for local learning

Author(s):

KAWAKITA, Masanori (The Graduate University of Advanced Studies);
EGUCHI, Shinto (The Graduate University of Advanced Studies)

Key words:

boosting; local likelihood; decision stump; VC dimension; classification.

Abstract:

    We propose a local boosting method in classification problems borrowing from an idea of the local likelihood method. The proposed method includes a simple device to localization for computational feasibility. We proved the Bayes risk consistency of the local boosting in the framework of PAC learning. Inspection of the proof provides a useful viewpoint for comparing the ordinary boosting and the local boosting with respect to the estimation error and the approximation error. Both boosting methods have the Bayes risk consistency if their approximation errors decrease to zero. Compared to the ordinary boosting, the local boosting may perform better by controlling the trade-off between the estimation error and the approximation error. Several numerical studies on real data sets confirm the advantageous aspects of the local AdaBoost over AdaBoost.