Introduction to Machine Learning

10-401, Spring 2018

Carnegie Mellon University

Maria-Florina Balcan

Previous material

This is a tentative schedule and is subject to change.

Date	Lecture	Topics	Readings and useful links	Handouts
Jan 17	Course Overview	Intro, admin Machine Learning Examples Decision Tree Learning		Slides
Jan 22	Decision Tree Learning	Decision Tree Learning The Big Picture Overfitting	Mitchell Chapters 1,2,6.1-6.3 Murphy Chapter 2 Bishop Chapter 1,2	Slides
Jan 24	Learning Linear Separators	Learning Linear Separators The Perceptron Algorithm Margins	Mitchell Chapters 4.1.2 and 4.4.1 Bishop Chapter 4.1.7 Daume: The Perceptron	Slides
Jan 29	Estimating Probabilities from Data	Bayes Rule MLE MAP	Mitchell: Estimating Probabilities	Slides
Jan 31	Naive Bayes	Conditional Independence Naive Bayes: Why and How	Mitchell: Naive Bayes and Logistic Regression	Slides
Feb 5	Naive Bayes	Naive Bayes: Why and How Bag of Words	Mitchell: Naive Bayes and Logistic Regression	Slides
Feb 7	Logistic Regression	Logistic Regression: Maximizing Conditional Likelihood Gradient Descent	Weinberger: Gradient Descent Mitchell: Naive Bayes and Logistic Regression	Slides
Feb 12	Logistic Regression		Mitchell: Naive Bayes and Logistic Regression Elkan Logistic Regression	Slides
Feb 14	Application Area: Computer Vision	Problems and Challenges in Computer Vision Deep Learning in Computer Vision	Lectures 8-11 from Jitendra Malik's course on computer vision	Slides
Feb 19	Kernels	Kernels Kernelizing Algorithms Kernelizing Perceptron	Bishop 6.1-6.2	Slides
Feb 21	Support Vector Machines	Geometric Margins SVM: Primal and Dual Forms Kernelizing SVM	Notes on SVM by Andrew Ng	Slides
Feb 26	Generalization and Overfitting	Sample Complexity Finite Hypothesis Classes	Mitchell: Ch 7 Notes on Generalization Guarantees	Slides
Feb 28	Generalization and Overfitting	Sample Complexity VC Dimension Based Bounds	Mitchell: Ch 7 Notes on Generalization Guarantees	Slides
Mar 5	Model Selection, Regularization			Slides
Mar 7	Midterm
Mar 12-16	No Class: Midsemester Break
Mar 19	Model Selection, Regularization	Structural Risk Minimization Regularization k-Fold Cross Validation		Slides Slides
Mar 21	Linear Regression	Linear Regression Minimizing squared error and maximizing data likelihood	Murphy: Chapter 7.1-7.3	Slides
Mar 26	Neural Networks	Neural Networks Backpropagation	Mitchell: Chapter 4	Slides
Mar 28	Deep Networks	Convolution Convolutional Neural Networks LeNet-5 Architecture	Goodfellow: Chapter 9	Slides
Apr 2	Boosting	Boosting Accuracy Adaboost	Shapire's NIPS Tutorial Shapire's Survey on Boosting	Slides
Apr 4	Unsupervised Learning	Objective Based Clustering Hierarchical Clustering	Hastie, Tibshirani and Friedman, Chapter 14.3 Center Based Clustering: A Foundational Perspective	Slides
Apr 9	Learning Representations Dimensionality Reduction	Hierarchical Clustering PCA Dimensionality Reduction	Bishop 12.1, 12.3	Slides Slides
Apr 11	Interactive Learning	Active Learning Common heuristics, Sampling bias Safe Disagreement Based Active Learning Schemes	Two Faces of Active Learning by Sanjoy Dasgupta	Slides Slides
Apr 16	Active Learning, Semi-Supervised Learning	Semi-Supervised Learning Transductive SVM Co-training	Semi-Supervised Learning in Encyclopedia of Machine Learning, Jerry Zhu	Slides Slides
Apr 18	Reinforcement Learning	Markov Decision Processes Value Iteration Q-Learning	Mitchell, Chapter 13 Kaelbling, et al., Reinforcement Learning: A Survey	Slides
Apr 23	Project Presentations
Apr 25	Project Presentations
Apr 30	Recap
May 2	Final