## Syllabus and (tentative) Course Schedule

Anouncements
Module 1: Supversived Learning
Thu 9/3 Lecture 1 (Eric, Ziv): Intro to probability, MLE - Slides
• Introduction of the course
• Basic probability
• Maximum likelihood estimate
Tue 9/8 No class
Thu 9/10 Lecture 2 (Ziv): Classification, kNN - Slides, Video
• Optimal decision using Bayes rule
• Types of classifiers
• Effect of values of k on kNN classifiers
• Probabilistic interpretation of kNN
Tue 9/15 No class
Thu 9/17 Lecture 3 (Ziv): Decision trees - Slides (updated), Video
• Discriminative classifiers
• Entropy
• Information gain
• Building decision trees
PS1 out
Tue 9/22 Lecture 4 (Ziv): Naïve Bayes - Slides, Quiz, Video
• Problems with estimating full joints
• Advantages of Naïve Bayes assumptions
• Applications to discrete and continuous cases
• Problems with Naïve Bayes classifiers
Mitchell, 6.1-6.10
Thu 9/24 Lecture 5 (Ziv): Linear regression - Slides, Video
• Basic model
• Solving linear regression
• Error in linear regression
Bishop, 3.1
Tue 9/29 Lecture 6 (Ziv): Logistic regression - Slides, Video
• Logistic regression vs. linear regression
• Sigmoid funcion
• Regularization
• Logistic regression for multiple classes
Bishop, 4.2-4.3
Thu 10/1 Lecture 7 (Eric): Perceptron, Neural networks - Slides, Video
• Perceptron
• Multilayer Perceptron
• Backpropagation
• "Deep" Learning
• Convolutional Neural Networks
• Layer-wise Pre-training
PS2 out,
PS1 due (10/2)
Tue 10/6 Lecture 8 (Eric): Deep learning, SVM - Slides1, Slides2, Video
• "Deep" Learning
• Convolutional Neural Networks
• Support Vector Machines
Deep Learning: SVM:
Thu 10/8 Lecture 9 (Eric): SVM - Slides, Annotated, Video
• Duality, KKT condition
• Kernel trick
• Sequential Minimal Optimization (SMO)
Proposal due
Tue 10/13 Lecture 10 (Eric): Evaluating classifiers, Bias-variance decomposition - Slides, Video
• Bias-variance decomposition
• Structural risk minimization
• Ways to avoid overfitting
Thu 10/15 Lecture 11 (Eric): Ensemble learning - Boosting, Random Forests - Slides, Video
• Combing weak learners
• Bagging and random forest
• AdaBoost, algorithem and generalization bounds
PS3 out,
PS2 due (10/16)
Module 2: Unsupversived Learning
Tue 10/20 Lecture 12, 13 (Ziv): Unsupervised learning - clustering - Slides, Video
• Hierarchical clustering
• K-means and Gaussian mixture models
• Number of clusters
• Bishop, Ch. 9
• Optional: Mitchell, 6.12
Thu 10/22
Tue 10/27 Lecture 14 (Ziv): Semi-supervised learning - Slides, Video
• Re-weighting
• EM, data augmentation
• Co-training
• Detect overfitting
Thu 10/29 Lecture 15 (Eric): Learning theory - Slides, Annotated, Video
• Realizable vs agnostic
• PAC learning in finite concept class
• Sample complexity
PS4 out,
PS3 due (10/30)
Tue 11/3 Lecture 16 (Eric): VC dimension - Slides, Video
• Sample complexity for infinite concept classes
• VC dimension as a complexity measure
• Structural risk minimization
• Ch. 3, An Introduction to Computational Learning Theory, M. Kearns and U. Vazirani
Module 3: Probabilistic Representation and Modeling
Thu 11/5 Lecture 17 (Eric): Graphical models, Bayes nets - Slides, Video Midway report due
Tue 11/10 Lecture 18 (Eric): Bayes nets - Slides, Video Bishop, 8.4
Thu 11/12 Lecture 19 (Eric): Undirected graphical models Video PS4 due (11/13)
Sun 11/15 Midterm review at Doherty 2315, 4-6pm
Tue 11/17 No class, Midterm at DH 2315, 5pm
Thu 11/19 Lecture 20 (Ziv): HMM - Slides, Video PS5 out
Tue 11/24 Lecture 21 (Ziv): HMM inference - Slides
Thu 11/26 No class
Tue 12/1 Lecture 22 (Eric): MDPs, Reinforcement learning - Slides
PS5 due
Thu 12/3 Lecture 23 (Eric): Topic models - Slides
Fri 12/4 Poster session, 2:30pm at NSH Atrium
Module 4: Applications of ML
Tue 12/8 Lecture 24 (Ziv): Computational biology - Slides Final report due (12/11)

© 2012 Eric Xing @ School of Computer Science, Carnegie Mellon University
[validate xhtml]