No. 1062

Title:

Information divergence geometry and the application to statistical machine learning

Author(s):

Eguchi, Shinto (The Institute of Statistical Mathematics)

Key words:

U-divergence, U-model, U-loss function, robustness, information geometry.

Abstract:

    The paper presents intuitive understandings for statistical learning from an information geometric point of view. We discuss a wide class of information divergence indices that express quantitatively a departure between any two probability density functions on a data space. In general the information divergence leads to a statistical method by minimization based on empirical data information. We introduce that the information divergence associates with a Riemannian metric and a pair of conjugate linear connections for a family of probability density functions. The most familiar example is the Kullback-Leibler divergence, which leads to the maximum likelihood method and associates with the information metric and the pair of the exponential and mixture connections. In the class of statistical methods led to by minimum divergence we discuss statistical properties focusing on robustness. As applications to statistical learning we discuss the minimum divergence methods for the principal component analysis, independent component analysis and statistical pattern recognition.