# Statistical Inference with Reproducing Kernel Hilbert Space

### 2008, Statistical Learning Theory II The Graduate University for Advanced Studies, ISM

Lecturer: Kenji Fukumizu(Institute of Statistical Mathematics)

Schedule: April.18 - Friday, 10:30-12:00

Place: (253(B) Kensyuu-shitsu, 2F)

## Class schedule

We have class on April 18, 25, May 2, 9, 23, 30, June 13, 20, 27. July 25, September 12, 26
No class on May 16, June 6, July 4, 11, 18.

## Purpose of course

An introduction to the methodology of statistical inferrence with positive definite kernels or reproducing kernel Hilbert spaces. The course explains the basic ideas of the methods with explanation of necessary mathematical background. Various kernel methods are explained, with special focus on SVM. Also, recent development of dependence anlaysis with kernels are explained in detail.

## Plan of lectures

#### 1. Introduction: overview of kernel methods (slides)

Basic idea of kernel method
Examples of kernel methods

#### 2. Elements of positive definite kernel and reproducing kernel Hilbert space (slides)

Definition and properties of positive definite kernel
Quick introduction to Hilbert spaces
Reproducing kernel Hilbert spaces

#### 3. Methods with kernels (slides)

Kernel PCA, Kernel CCA, Kernel FDA, SVM
Representer theorem

#### 4. Support vector machines I (slides)

A quick course on convex optimization
Optimization in learning of SVM

#### 5. Support vector machines II (slides)

Multiclass classification
Structured output
Variants of SVM

#### 6. Generalization ability of SVM (slides)

Basics of computational learning theory
generalization ability of SVM

#### 7. Theory of positive definite kernel and reproducing kernel Hilbert space (slides)

Negative definite kernel and Sch"onberg's theorem
Various examples of positive definite kernels
Bochner's theorem
Hilbert-Schmidt operator
Mercer's theorem

#### 8. Mean Element in RKHS (slides)

Mean in RKHS
Characteristic kernel
Homogeneity test with kernels

#### 9. Dependence analysis with kernels (slides)

Dependence and independence with kernels
Conditional independence with kernels
Application to dimension reduction

#### 10. Various aspects of kernel methods (slides)

Relation to Gaussian process
Relation to smoothing spline
Functional data alalysis

## References

• Learning with Kernels. B.Schoelkopf and A.Smola. (2001) MIT Press.
• An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. N.Cristianini and J.Shawe-Taylor (2000) Cambridge University Press.

## Evaluation

Report topics will be assgined during the course.

Back to Kenji Fukumizu's home page

Back to ISM