Advanced Machine Learning Eric Xing |
Lectures
1 |
Introduction: Slides |
Models Functional approximation, Multivariate statistical models, Supervised versus unsupervised, High-dimensional inference, Applications |
2 |
Univariate prediction without
using a model: good or bad? Slides |
kNN, spectrum graph methods, |
3 |
Data visualization and
dimensionality reduction Slides |
PCA manifold learning |
4 |
Learning Unstructured Predictive
Models: Generative versus Discriminative Slides |
Gaussian discrimination linear and logistic regression |
5 |
Mixture Models: EM Slides |
Maximum likelihood est. versus
Bayesian est. Partially observed model The EM algorithm Theory of EM |
6 |
Support Vector Machine: optimize
predictive margin Slides |
Maximum margin learning Lagrangian duality Convex optimization Kernel trick |
7 |
Learning Theory Slides |
PAC learning theory VC dimension Structural Risk Minimization: |
8 |
Overfitting, Regularization,
Model Selection Slides |
Overfitting Decomposition of error into bias and variance Lasso Model selection consistency |
9 |
Structured Models: Hidden Markov
Models versus Conditional Random Fields Slides |
HMM versus CRF Forward-Backward algorithm |
10 |
Graphical Models: Bayesian
network versus Markov random fields Slides |
Representations
independences properties, I-map Equivalence theorem The Bayes-ball algorithm |
11 |
Exact Inference Slides |
Message-passing
on trees and on the original graph Junction tree algorithm |
12 |
Variational Inference Slides |
Theory of
loopy belief propagation Theory of mean field inference |
13 |
Markov Chain Monte Carlo Slides |
Metropolis
Hasting algorithm Gibbs sampling algorithm, convergence test; The data augmentation algorithm and EM. |
14 |
Learning fully observed and
partially observed BN Slides |
EM revisit |
15 |
Maximum likelihood learning of
undirected GM Slides |
CRFs and
general Markov networks Iterative scaling learning Contrastive divergence algorithms |
16 |
Max-margin learning of GM Slides |
Structured
input-output models Max-margin Markov networks |
17 |
Infinite-dimensional models: the
Dirichlet process Slides |
nonparametric Bayesian DP and DP mixture Chinese restaurant process |
18 |
Application 1: Text and Image
information retrieval Slides |
Topic models |
19 |
Application 2: Genetics Slides |
Genome association via
multivariate regression Haplotype inference via Dirichlet process |
20 |
Application 3: Social and
Biological Networks Slides |
Time-varying network learning exponential random graph models mixed membership stochastic models |
© 2009 Eric Xing @ School of Computer Science, Carnegie Mellon University