ISM Research Memorandum
No. 1006
Title:
A bridge between boosting and a kernel machine
Author(s):
KAWAKITA, Masanori (The Institute of Statistical Mathematics);
IKEDA,
Shiro (The Institute of Statistical Mathematics);
EGUCHI, Shinto (The
Institute of Statistical Mathematics)
Key words:
boosting; kernel machines; regularization; gram matrix.
Abstract:
In this paper, boosting methods are studied from a
viewpoint of kernel machines. This natural connection has already been
revealed by defining a kernel function associated with the set of weak
learners, which we call the WL kernel (Weak Learner kernel). We review this
connection with respect to a kernel exponential family, and propose two
important extensions of boosting methods for classification problems. First
proposal is a new simple regularized boosting, which is confirmed to be valid
through some experiments on real data. The other is a new simple kernel
function from the investigation of the RKHS of decision stumps, which is one
of the most widely-used weak learners. Several experiments confirm the
efficiency and the validity of the proposed algorithm with the new kernel
function.