Course Description

Date	Lecture	Scribe	Topics	Readings	Handouts
Mon 14th Jan	Lecture 1 (Eric): Introduction to Graphical Models; Directed GM: Bayesian Networks Slides			Required: Koller and Friedman: Chpt. 1 An Introduction to Graphical Models	Scribe Template
Module 1: Representation
Wed 16th Jan	Lecture 2 (Eric): Undirected GM: Markov Random Fields Slides	Chunlei Liu (andrew id:chl56) Ahmed Hefny (ahefny) Scribe Notes		Required: Koller and Friedman: Chpt 3 (The Bayesian Network Representation) Koller and Friedman: Chpt 4 (Undirected Graphical Models) Optional: Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Mon 21st Jan	No class
Wed 23rd Jan	Lecture 3 (Eric): A unified view of BN and MRF Slides	Zhen Tang (zhent) William Chan (wchan2) Scribe Notes		Required: Koller and Friedman: Chpt 4.5 Optional: Pseudo-Likelihood Based Structure Estimation using Neighborhood Estimation: Neighborhood Selection in Gaussian Graphical Models Likelihood Based Structure Estimation of GGMs: Glasso
Module 2: Basic Inference and Learning Methods
Mon 28th Jan	Lecture 4 (Eric): Exact Inference (1): Variable Elimination Slides	Georg Schonherr (gschoenh) Micol Marchetti-Bowick (mmarchet) Scribe Notes		Required: Koller and Friedman, Chpt 9.1, 9.2, 9.3, 9.4
Wed 30th Jan	Lecture 5 (Eric): Exact Inference (2): Belief Propagation Slides	Da-Cheng Juan (djuan) Jinliang Wei (jinlianw) Scribe Notes		Required: Koller and Friedman Chpt. 10.1, 10.2, 10.3 Optional: A useful tutorial is here.
Mon 4th Feb	Lecture 6 (Eric): Generalized Linear Models and sufficient statistics Slides	Thom Popovici (dpopovic) Huan-Kai Peng (huankaip) Scribe Notes		Required: Koller and Friedman: Chpt 17.2, 17.3, 17.4 (Particularly 17.4.4: MAP Estimation) Optional: Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions A Characterization of the Dirichlet Distribution Through Global and Local Parameter Independence
Wed 6th Feb	Lecture 7 (Eric): Learning (1): Fully observed BN Slides	Milad Memarzadeh (mmemarza) Shen, Xiaolong (xiaolons) Scribe Notes		Required: Koller and Friedman: Chpt 17.1, 17.3 Optional: Structural learning using neighborhood selection:High-dimensional graphs and variable selection with the Lasso Chow-Liu tree seminal paper:Approximating discrete probability distributions with dependence trees
Mon 11th Feb	Lecture 8 (Eric): Learning (2): Fully observed MRF Annotated Slides	Rishi Chandy (rishic) AJ Sedgewick (asedgewi) Scribe Notes		Required: Jordan: Chpt 9, 20 Optional: Koller and Friedman: Chpt 20.1, 20.2, 20.3 Graphical lasso
Wed 13th Feb	Lecture 9 (Eric): Learning (3): EM algorithm Slides	Mrinmaya Sachan (mrinmays) Phani Gadde (pgadde) Viswanathan, Sripradha (sripradv) Scribe Notes		Required: Koller and Friedman: Chpt. 19.1, 19.2.2, 19.2.3 Optional: Some interesting aspects of EM
Module 3 : Case Studies I: Popular Graphical Models
Mon 18th Feb	Lecture 10 (Eric): Gaussian graphical models and Ising models: modeling networks Slides	Kartik Goyal (kartikgo) Scribe Notes		Required: Covariance Selection - the original GGM paper by Dempster Glasso Highly Recommended: Estimating Time-Varying Networks, Annals of Applied Statistics, Vol. 4, No. 1, 94 - 123, 2010 (earlier version). TREEGL: reverse engineering tree-evolving gene networks underlying developing biological lineages Optional: Meinshausen-Buhlmann algorithm Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian
Wed 20th Feb	Lecture 11 (Eric): Factor Analysis and State-Space Models Slides	Xuezhi Wang (xuezhiw) Adona Iosif (aiosif) Scribe Notes		Required: Koller and Friedman: Chpt 15.4 An introduction to the Kalman filter Highly Recommended: Factor Analysis Dynamic Topic Models Optional: Variational Learning for Switching State-Space Models A discrete state-space model for linear image processing
Mon 25th Feb	Lecture 12 (Gunhee): Conditional Random Fields Case study: image segmentation in computer vision Slides	Salim Akhter Chowdhury (sachowdh) Jingwei Zhang (jingweiz) Scribe Notes		Required: A tutorial on HMMs CRF Tutorial by Hanna Wallach Highly Recommended: Discriminative Random Fields. Sanjiv Kumar and Martial Hebert. Structured Prediction and Learning in Computer Vision. CVPR 2011 Tutorial. Optional: The original CRF paper Shallow Parsing with CRFs
Module 4 : Approximate Inference
Wed 27th Feb	Lecture 13 (Eric): Variational Inference: Loopy Belief Propagation and Mean Field Approximation Slides	Peter F. Schulam (pschulam) William Y. Wang (yww) Scribe Notes		Required: M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul. An introduction to variational methods for graphical models E. P. Xing, M. I. Jordan, and S. Russell A generalized mean field algorithm for variational inference in exponential families Highly Recommended: Murphy K., Weiss Y. and Jordan M. Loopy belief propagation for approximate inference: an empirical study Optional: Wainwright and Jordan: Chpt 5 Koller and Friedman: Chpt 11
Mon 4th Mar	Lecture 14 (Eric/Chong): Variational Inference: Mean Field Approximation Case study: learning topic models Slides	Miaomiao Wen (mwen) Lingpeng Kong (lingpenk) Scribe Notes		Required: Wainwright & Jordan. Variational inference in graphical models: The view from the marginal polytope Wainwright and Jordan: Chpts 3, 5 Highly Recommended: D. M. Blei, A. Y. Ng, M. I. Jordan Latent Dirichlet Allocation A. Ahmed and E. P. Xing On Tight Approximate Inference of Logistic-Normal Admixture Model M. Hoffman, D. M. Blei, C. Wang, J. Paisley Stochastic Variational Inference
Wed 6th Mar	Lecture 15 (Junming): Theory of variational inference: inner and outer approximation Slides	Ankur Gandhe (ankurgan) Wei Zhang (weizhan1) Scribe Notes		Required: Wainwright and Jordan: Chpt 3.4-3.6, 4.1, 5.1-5.3.
Mon 11th Mar	No class
Wed 13th Mar	No class
Mon 18th Mar	Lecture 16 (Eric): Monte Carlo Methods Slides	Willie Neiswanger (wdn) Xiaohua Yan (xiaohuay) Scribe Notes		Required: Mackay: Chpt 29.1 - 29.3 Optional: Mackay. Introduction to Monte Carlo Methods
Wed 20th Mar	Lecture 17 (Qirong): Markov Chain Monte Carlo Case study: learning topic models Slides	Dougal Sutherland (dsutherl) Keerthiram Murugesan (kmuruges) Zhang, Ada June (ajzhang) Scribe Notes		Required: Mackay: Chpt 29.4 - 29.10 Optional: Mackay. Introduction to Monte Carlo Methods
Mon 25th Mar	Lecture 18 (Qirong): Advanced topics in MCMC Slides	Dougal Sutherland (dsutherl) Keerthiram Murugesan (kmuruges) Zhang, Ada June (ajzhang) Scribe Notes		Required: Hastie and Green. Model Choice using Reversible Jump Markov Chain Monte Carlo Doucet and Johansen. A Tutorial on Particle Filtering and Smoothing: Fifteen years later: Chpt 3
Module 5 : Kernel Graphical Models and Spectral Methods
Wed 27th Mar	Lecture 19 (Ankur): Hilbert Space Embeddings of Distributions Slides	Chenyan Xiong (cx) Dena Marie Asta (dasta) Xu, Felix (juefeix) Scribe Notes		Required: Chpt. 2.1, 2.2, 3.1, 3.2 L. Song. Learning via Hilbert space embedding of distributions Optional: L. Song, J. Huang, A. Smola, and K. Fukumizu. Hilbert Space Embedding of Conditional Distributions with Applications to Dynamical Systems
Mon 1th Apr	Lecture 20 (Ankur): Kernel Graphical Models Slides	Gupta, Anika (anikag) Lan, Zhen-Zhong (lanzhzh) Scribe Notes		Required: L. Song, J. Huang, A. Smola, and K. Fukumizu. Hilbert Space Embedding of Conditional Distributions with Applications to Dynamical Systems Optional: L. Song, A. Gretton, and C. Guestrin. Nonparametric Tree Graphical Models via Kernel Embeddings L. Song, A. Gretton, D. Bickson, Y. Low, and C. Guestrin.Kernel Belief Propagation
Wed 3rd Apr	Lecture 21 (Ankur): Spectral Algorithms for Graphical Models Case Study: Sentence Parsing in Natural Languages Slides	Kenton Murray (andrew id: kwmurray) Shashank (shashans) Scribe Notes		Required: D. Hsu, S. Kakade, and T. Zhang. A Spectral Algorithm for Learning Hidden Markov Models Optional: A.P. Parikh, L. Song, and E.P. Xing. A Spectral Algorithm for Latent Tree Graphical Models L. Song, B. Boots, S. Siddiqi, G. Gordon, and A. Smola.Hilbert Space Embeddings of Hidden Markov Models L. Song, A.P. Parikh, and E.P. Xing.Kernel Embeddings of Latent Tree Graphical Models S. Cohen, K. Stratos, M. Collins, D. Foster, L. Ungar. Spectral Learning of Latent-Variable PCFGs
Module 6 : Nonparemetric Bayesian Models and "Infinite" Models
Mon 8th Apr	Lecture 22 (Sinead): Dirichlet Process Slides	Abhimanu Kumar (abhimank) Victor Chahuneau (vchahune) Rory Donovan (donovanr) Scribe Notes		Required: Yee Whye Teh. Dirichlet Process Optional: Carl Edward Rasmussen. Gaussian processes for machine learning
Wed 10th Apr	Lecture 23 (Sinead): Indian Buffet Process Slides	Palakodety, Shriphani (spalakod) Scribe Notes		Required: Thomas L. Griffiths and Zoubin Ghahramani. The Indian Buffet Process: An Introduction and Review
Mon 15th Apr	Lecture 24 (Sinead/Eric): Hierarchical Dirichlet Process Case Study: Genetic Inference of the World Population Slides 1 Slides 2	Zaid Sheikh (zsheikh) Alex Michael Beutel (abeutel) Scribe Notes		Required: K. Sohn, E. P. Xing. A hierarchical Dirichlet process mixture model for haplotype reconstruction from multi-population data
Module 7 : Structured Sparsity
Wed 17th Apr	Lecture 25 (Eric): Graphical induced structured input/output models Case Study: Disease Association Analysis Slides	Meghana Kshirsagar (mkshirsa) Yiwen Chen (yiwenche) Scribe Notes		Required: X. Chen et al. Smoothing Proximal Gradient Method for General Structured Sparse Regression S. Kim, E. P. Xing. Statistical Estimation of Correlated Genome Associations to a Quantitative Trait Network Optional: S. Kim, E. P. Xing. Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity
Wed 22nd Apr	Lecture 26 (Seunghak): Convex Optimization Slides	Sashank J Reddi (sjakkamr) Chen, Yun-Nung (yvchen)		Required: F. Bach et al. Convex Optimization with Sparsity-Inducing Norms Highly Recommended: F. Bach, R. Jenatton, J. Mairal and G. Obozinski Optimization with Sparsity-Inducing Penalties
Mon 24th Apr	Lecture 27 (Eric): Structured Sparse Additive Models Slides	Wei Dai (wdai) Cong Lu (congl) Wang, Guanyu (guanyuw) Scribe Notes		Required: Ravikumar, P., Lafferty, J., Liu, H., and Wasserman, L. Sparse additive models Yin, J., Chen, X., and Xing, E. Group Sparse Additive Models Highly Recommended: Hastie, T., and Tibshirani, R. Generalized Additive Models
Module 8 : Posterior Regularization and Max-Margin GM:
Mon 29th Apr	Lecture 28 (Eric): Posterior Regularization and The Maximum Entropy Discrimination Principle Slides	Cheng, Andrew (ac1) Downey, Carlton MacDonald (cmdowney) Scribe Notes		Required: B. Taskar, C. Guestrin, D. Koller Max-margin Markov networks J. Zhu, E. P. Xing On Primal and Dual Sparsity of Markov Networks
Wed 1st May	Lecture 29 (Eric): Maximum margin GM Slides Case Study: discriminative topic models for text and image	Fahrenkopf, Max A (mfahrenk) Gong, Yu (ygong1)		Required: J. Zhu, A. Ahmed, E. P. Xing MedLDA: Maximum Margin Supervised Topic Models for Regression and Classi.cation J. Zhu, N. Chen, E. P. Xing Bayesian Inference with Posterior Regularization and applications to Infinite Latent SVMs
Mon 6th May	Class presentation Completed Projects NSH 1305 from 9AM to 1PM

 

© 2009 Eric Xing @ School of Computer Science, Carnegie Mellon University
[validate xhtml]