Advanced Introduction to Machine Learning
10715, Fall 2014Eric Xing, Barnabas Poczos School of Computer Science, CarnegieMellon University 
Syllabus and (tentative) Course Schedule
Date  Lecture  Topics  Readings and useful links 
Anouncements 

Block 1:  Supervised Learning  
Mon 9/8  Lecture 1: Regression: Linear and Logistic Eric Xing: slides, annotated slides 
No Reading assignment due Bishop, PRML: Ch 4, Ch 5 Mitchell: Ch 4 Chapter (Draft) from Mitchell On Discriminative and Generative Classifiers Ng, Jordan 

Wed 9/10  Lecture 2: Linear Regression and Lasso Eric Xing: slides annotated slides 
Tutorial on Regression by Andrew Moore Bishop, PRML: Ch 3 Mitchell: Ch 8.3 Regression Shrinkage and Selection via the Lasso by Rob Tibshirani Model Selection and Estimation in Regression with Grouped Variables by Yuan, Lin Large Scale Online Learning by Bottou, Le Cun Feature Selection for HighDimensional Genomic Microarray Data by Xing, Jordan, Karp On Model Selection Consistency of Lasso by Peng Zhao, Bin Yu 

Mon 9/15  Lecture 3: Structured sparsity with application in Computational Genomics Eric Xing: slides 
Statistical Estimation of Correlated Genome Associations to a Quantitative Trait Network by S. Kim and E. P. Xing TreeGuided Group Lasso for MultiResponse Regression with Structured Sparsity, with applications to eQTL Mapping by S. Kim and E. P. Xing Smoothing Proximal Gradient Method for General Structured Sparse Regression by X. Chen, Q. Lin, S. Kim, J. Carbonell and E. P. Xing 

Wed 9/17  Lecture 4: Perceptron, Deep Neural Networks Barnabas Poczos: MultiLayerPerceptron DeepArchitectures 
Learning Deep Architectures for AI by Yoshua Bengio ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky et. al. Multilayer Feedforward Networks are Universal Approximators by Kur Hornik A Logical Calculus of the ideas immanent in Nervous Activity by Warren S. McCulloch, Walter Pitts The Perceptron: A probabilistic model for information storage and organization in the brain by F. Rosenblatt Optional: Some slides by Eric on Learning DNNs. 
Homework 1 out 

Block 2:  Kernel Machines  
Mon 9/22 
Lecture 5: SVMs and Duality Barnabas Poczos: SupportVectorMachines Duality 


Wed 9/24  Lecture 6: The Kernel Trick & RKHS Eric Xing: slides annotated slides 
Required Learning with Kernels, Scholkof & Smola, Ch 2 

Mon 9/29  Lecture 7: Reproducing Kernel Hilbert Space
Eric Xing 

Wed 10/1  Lecture 8: Learning with Kernels
Barnabas Poczos 
Homework 1 due Homework 2 out 

Block 3:  Unsupervised Learning, Density estimation, Graphical Models  
Mon 10/6  Lecture 9: Clustering, mixture models, the EM algorithm
Barnabas Poczos: slides 
Required Max Welling's notes on Clustering and EM. 

Wed 10/8  Lecture 10: Clustering, mixture models, the EM algorithm
Barnabas Poczos: Slides same as above 

Mon 10/13  Lecture 11: Structured Models: Hidden Markov Models vs. Conditional Random Fields Eric Xing: slides 
Required Chap. 12 from Michael Jordan's book Chap 12 Shallow Parsing with Conditional Random Fields Optional Rabiner, Lawrence R. (1989). A Tutorial on Hidden Markov Model and selected Applications in Speech Recognition Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data 

Wed 10/15  Lecture 12: Structured Models: Hidden Markov Models vs. Conditional Random Fields, Graphical Models Eric Xing: CRF GraphicalModels 
Required Michael Jordan's Introduction to Graphical Models 
Homework 2 due 

Mon 10/20  Lecture 13: Graphical Models, Markov Chain Monte Carlo and Topic Models
Eric Xing: slides 
Required


Wed 10/22  Lecture 14: Markov Chain Monte Carlo
Eric Xing: slides (cont'd) 

Mon 10/27  Mid Term  
Block 4:  Latent Space Analysis, Eigen space analysis  
Wed 10/29 
Lecture 15: Principal Component Analysis Barnabas Poczos: slides 
Required

Homework 3 out 

Mon 11/3 
Lecture 16: Independent Component Analysis Barnabas Poczos: slides 
Required


Wed 11/5 
Lecture 17: Independent Component Analysis Barnabas Poczos: slides continued from previous lecture  
Block 5:  Bayesian Nonparametrics  
Mon 11/10 
Lecture 18: Gaussian Processes Barnabas Poczos: slides 
Required


Wed 11/12 
Nonparametric Bayesian Models Eric Xing: slides 
Required

Homework 3 due Homework 4 out (Fri 11/14) 

Mon 11/17 
Spectral clustering Eric Xing: slides 
Required  

Block 6:  Computational Learning theory  
Wed 11/19 
Risk Minimization Barnabas Poczos: slides 
Required

Homework 4 due (Fri 11/21) 

Mon 11/24 
VC theory Barnabas Poczos slides 

Wed 11/26  Thanks Giving Holiday  

Mon 12/1 
Manifold Learning Barnabas Poczos slides 
Required:


Block 7:  Ensemble methods  
Wed 12/3  Boosting, random forests  

Additional:  If we have more time  
Online Learning  

Local linear embedding, and manifold learning  
© 2012 Eric Xing @ School of Computer Science, Carnegie Mellon University
[validate xhtml]
[validate xhtml]