# Statistical Inference with Reproducing Kernel Hilbert Space

### 2008, Statistical Learning Theory II

The Graduate University for Advanced Studies, ISM

Lecturer:
Kenji Fukumizu(Institute of Statistical Mathematics)

Schedule: April.18 - Friday, 10:30-12:00

Place: (253(B) Kensyuu-shitsu, 2F)

## Class schedule

We have class on April 18, 25, May 2, 9, 23, 30, June 13, 20, 27. July 25, September 12, 26

No class on May 16, June 6, July 4, 11, 18.

## Purpose of course

An introduction to the methodology of statistical inferrence with positive definite
kernels or reproducing kernel Hilbert spaces. The course explains the basic
ideas of the methods with explanation of necessary mathematical background.
Various kernel methods are explained, with special focus on SVM. Also, recent development
of dependence anlaysis with kernels are explained in detail.

## Plan of lectures

#### 0. Outline and Information on the course
(slides)

#### 1. Introduction: overview of kernel methods
(slides)

Basic idea of kernel method

Examples of kernel methods

#### 2. Elements of positive definite kernel and reproducing kernel
Hilbert space (slides)

Definition and properties of positive definite kernel

Quick introduction to Hilbert spaces

Reproducing kernel Hilbert spaces

#### 3. Methods with kernels (slides)

Kernel PCA, Kernel CCA, Kernel FDA, SVM

Representer theorem

#### 4. Support vector machines I
(slides)

A quick course on convex optimization

Optimization in learning of SVM

#### 5. Support vector machines II (slides)

Multiclass classification

Structured output

Variants of SVM

#### 6. Generalization ability of SVM (slides)

Basics of computational learning theory

generalization ability of SVM

#### 7. Theory of positive definite kernel and reproducing kernel
Hilbert space (slides)

Negative definite kernel and Sch"onberg's theorem

Various examples of positive definite kernels

Bochner's theorem

Hilbert-Schmidt operator

Mercer's theorem

#### 8. Mean Element in RKHS (slides)

Mean in RKHS

Characteristic kernel

Homogeneity test with kernels

#### 9. Dependence analysis with kernels (slides)

Dependence and independence with kernels

Conditional independence with kernels

Application to dimension reduction

#### 10. Various aspects of kernel methods (slides)

Relation to Gaussian process

Relation to smoothing spline

Functional data alalysis

## References

- Learning with Kernels. B.Schoelkopf and A.Smola. (2001) MIT Press.
- An Introduction to Support Vector Machines and Other Kernel-based Learning Methods.
N.Cristianini and J.Shawe-Taylor (2000) Cambridge University Press.

## Evaluation

Report topics will be assgined during the course.

Back to Kenji Fukumizu's home page

Back to ISM