10701/15781 Machine Learning

Lectures

1	Introduction: Slides	Models Functional approximation, Multivariate statistical models, Supervised versus unsupervised, High-dimensional inference, Applications
2	Univariate prediction without using a model: good or bad? Slides	kNN, spectrum graph methods,
3	Data visualization and dimensionality reduction Slides	PCA manifold learning
4	Learning Unstructured Predictive Models: Generative versus Discriminative Slides	Gaussian discrimination linear and logistic regression
5	Mixture Models: EM Slides	Maximum likelihood est. versus Bayesian est. Partially observed model The EM algorithm Theory of EM
6	Support Vector Machine: optimize predictive margin Slides	Maximum margin learning Lagrangian duality Convex optimization Kernel trick
7	Learning Theory Slides	PAC learning theory VC dimension Structural Risk Minimization:
8	Overfitting, Regularization, Model Selection Slides	Overfitting Decomposition of error into bias and variance Lasso Model selection consistency
9	Structured Models: Hidden Markov Models versus Conditional Random Fields Slides	HMM versus CRF Forward-Backward algorithm
10	Graphical Models: Bayesian network versus Markov random fields Slides	Representations independences properties, I-map Equivalence theorem The Bayes-ball algorithm
11	Exact Inference Slides	Message-passing on trees and on the original graph Junction tree algorithm
12	Variational Inference Slides	Theory of loopy belief propagation Theory of mean field inference
13	Markov Chain Monte Carlo Slides	Metropolis Hasting algorithm Gibbs sampling algorithm, convergence test; The data augmentation algorithm and EM.
14	Learning fully observed and partially observed BN Slides	EM revisit
15	Maximum likelihood learning of undirected GM Slides	CRFs and general Markov networks Iterative scaling learning Contrastive divergence algorithms
16	Max-margin learning of GM Slides	Structured input-output models Max-margin Markov networks
17	Infinite-dimensional models: the Dirichlet process Slides	nonparametric Bayesian DP and DP mixture Chinese restaurant process
18	Application 1: Text and Image information retrieval Slides	Topic models
19	Application 2: Genetics Slides	Genome association via multivariate regression Haplotype inference via Dirichlet process
20	Application 3: Social and Biological Networks Slides	Time-varying network learning exponential random graph models mixed membership stochastic models