Computational methodology in statistical inference II
FY2008, The Graduate University for Advanced Studies, ISM
Lecturer: Kenji Fukumizu (The Institute of Statistical Mathematics)
Schedule: Jan.7-Feb.25 Wednesday, 10:30-12:00
Place: Kensyu-shitsu
Purpose of course
An introduction to the method of graphical models, which represent
probabilistic relations among variables using directed, undirected, or
factor graphs. Methods of inference (belief propagation and related
ones) and learning (parameter estimation), and approximation are
discussed.
Questions for Report
To be given.
Plan of lectures
1. Introduction to Graphical Models. slides of part 1
What are graphical models?
Graphs and probabilities -- Factorization and Markov properties
Undirected graph, directed graph, factor graph
2. Mixture Models and Hidden Markov Models slides of part 2
Mixture model
Hidden Markov model
3. Inference with Graphical Models -- propagation algorithm slides 3-(a),
slides 3-(b)
Belief propagation algorithm for trees
Application of BP to HMM
Inference for non-tree graphs (Junction tree, loopy BP)
General propagation algorithm (max-product etc.)
4. Learning of Graphical Models -- parameter estimation and structure learning
slides 4-(a), slides 4-(b)
ML approach to parameter estimation: EM algorithm for hidden variables
EM for mixture models
Bayesian approach to parameter estimation
Structure learning and causal inference
5. Approximate Inference slides 5
Brief surveys on various methods for approximate inference
Variational Bayes
Sampling (Importance sampling, MCMC: very quick survey)
References
Pattern Recognition and Machine Learning. Christopher M. Bishop. Springer (2006) Chaps.8-10, 13.
Graphical Models. S.L. Lauritzen. Oxford University Press (1996).
Probabilistic Networks and Expert Systems. R.G. Cowell, A.P. Dawid, S.L. Lauritzen, and D.J. Spiegelhalter. Springer (1999)
Bayesian Networks and Decision Graphs (2nd ed.). Finn V. Jensen and Thomas D. Nielsen. Springer (2007)
「統計的因果推論」宮川雅巳.朝倉書店(2004)
この講義とは異なり、因果推論の立場からグラフィカルモデルを論じている。
「統計科学のフロンティア11, 計算統計I」平均場近似・EM法・変分ベイズ法.樺島・上田.岩波書店(2003)
変分ベイズ法のよい解説。
Evaluation
Report topics will be assgined during the course.
Back to Kenji Fukumizu's home page
Back to ISM