Computational methodology in statistical inference II

FY2008, The Graduate University for Advanced Studies, ISM

Lecturer: Kenji Fukumizu (The Institute of Statistical Mathematics)

Schedule: Jan.7-Feb.25 Wednesday, 10:30-12:00

Place: Kensyu-shitsu

Purpose of course

Questions for Report

Plan of lectures

1. Introduction to Graphical Models. slides of part 1

What are graphical models?
Graphs and probabilities -- Factorization and Markov properties
Undirected graph, directed graph, factor graph

2. Mixture Models and Hidden Markov Models slides of part 2

Mixture model
Hidden Markov model

3. Inference with Graphical Models -- propagation algorithm slides 3-(a), slides 3-(b)

Belief propagation algorithm for trees
Application of BP to HMM
Inference for non-tree graphs (Junction tree, loopy BP)
General propagation algorithm (max-product etc.)

4. Learning of Graphical Models -- parameter estimation and structure learning slides 4-(a), slides 4-(b)

ML approach to parameter estimation: EM algorithm for hidden variables
EM for mixture models
Bayesian approach to parameter estimation
Structure learning and causal inference

5. Approximate Inference slides 5

Brief surveys on various methods for approximate inference
Variational Bayes
Sampling (Importance sampling, MCMC: very quick survey)

References

Pattern Recognition and Machine Learning. Christopher M. Bishop. Springer (2006) Chaps.8-10, 13.
Graphical Models. S.L. Lauritzen. Oxford University Press (1996).
Probabilistic Networks and Expert Systems. R.G. Cowell, A.P. Dawid, S.L. Lauritzen, and D.J. Spiegelhalter. Springer (1999)
Bayesian Networks and Decision Graphs (2nd ed.). Finn V. Jensen and Thomas D. Nielsen. Springer (2007)

「統計的因果推論」宮川雅巳．朝倉書店(2004)
この講義とは異なり、因果推論の立場からグラフィカルモデルを論じている。

「統計科学のフロンティア11, 計算統計Ｉ」平均場近似・ＥＭ法・変分ベイズ法．樺島・上田．岩波書店(2003)
変分ベイズ法のよい解説。

Evaluation

Report topics will be assgined during the course.