Course Overview

In this course we will introduce classical methods in pattern classification and machine learning, focusing on statistical learning approaches for supervised and unsupervised learning problems. We will present the theoretical and algorithmic underpinnings of these methods. The course will consist of biweekly lectures, problem sets that contain both mathemetical and MATLAB/Octave programming exercises, and two in-class exams.


Undergraduate level training or coursework in algorithms, linear algebra, calculus and multivariate calculus, basic probability and statistics; an undergraduate level course in Artificial Intelligence may be helpful but is not required. A background in programming will also be necessary for the problem sets; specifically students are expected to be familiar with MATLAB/Octave or learn it during the course.

Piazza Forum

We will use Piazza for class discussions. Please go to this Piazza website to join the course forum (note: you must use a email account to join the forum). We strongly encourage students to post on this forum rather than emailing the course staff directly (this will be more efficient for both students and staff). Students should use Piazza to:

The course Academic Integrity Policy must be followed on the message boards at all times. Do not post or request homework solutions! Also, please be polite.

Staff Contact Info

Instructor: Prof. Ameet Talwalkar
Office Hours: Monday after class, Boelter 4531F
Email: ameet at

Teaching Assistants:


There will be no required textbooks, though we suggest the following to help you to study: We will provide suggested readings from these books in the schedule below, using the acronyms MLAPA and ESL.

Academic Integrity Policy

Group studying and collaborating on problem sets are encouraged, as working together is a great way to understand new material. Students are free to discuss the homework problems with anyone under the following conditions: Students are encouraged to read the Dean of Students' guide to Academic Integrity.

Grading Policy

Grades will be based on the following components:

Regrade requests for homework or exams must be made within one week after the graded assignments have been handed out, regardless of whether you attended class that day. Please put your request in writing, indicating the parts you want regraded and why, and hand in your request to the TA in person. Please keep in mind the unlikely but real possibility that a regrade may lower your grade, in case a previously overlooked mistake is discovered.

Using MATLAB / Octave

All registered students for this course can obtain MATLAB access via their SEASnet accounts. This applies to both Engineering and non-Engineering students), though students who are not officially enrolled (per the Registrar's Office) will not be able to get a SEAS account. SEASnet accounts can be applied for by going to this link, then clicking on the "Create Account" tab in the upper left hand corner. Student accounts must be picked up in person at the SEASnet Help Desk. Remote Desktop is also available to all enrolled students in Engineering. If you need assistance please contact

Octave is freely available.

Using LaTeX

Students are strongly encouraged to use LaTeX. LaTeX makes it simple to typeset mathematical equations, and is extremely useful for grad students to know. Most of the academic papers you read were written with LaTeX, and probably most of the textbooks too. Here is an excellent LaTeX tutorial and here are instructions for installing LaTeX on your machine. Also note that LaTeX is also installed on department-run linux machines.


This course is based in part on material developed by Fei Sha. Some of the administrative content on the course website is adapted from material from Jenn Wortman Vaughan, Rich Korf, and Alexander Sherstov.


Date Topics Reading HW
1/9 Course Overview, Math Review [MLAPA] 1.1-1.3, 2 HW1 released (supplementary files)
1/11 Nearest Neighbors

[MLAPA] 1.4.1-1.4.3;
[ESL] 13.3
1/16 No Class -- MLK Day

Decision Trees

[MLAPA] 16.2; [ESL] 9.2 HW1 due

Naive Bayes

[MLAPA] 3.5; [ESL] 6.6.3 HW2 released (supplementary file)

Logistic Regression

[MLAPA] 1.4.6, 8.1-8.4;
[ESL] 4.1, 4.2, 4.4

Gaussian and Linear Discriminant Analysis, Multiclass Classification

[MLAPA] 4.2.1-4.2.5;
[ESL] 4.3

Perceptron, Linear Regression

[MLAPA] 8.5, 1.4.5, 7.1-7.3, 7.5.1, 7.5.2, 7.5.4,7.6;
[ESL] 3.1-3.2
HW2 due;
HW3 released (supplementary file)

Linear Regression, Nonlinear Basis Functions

[MLAPA] 1.4.7, 1.4.8;
[ESL] 7.1-7.3, 7.10
Linear Algebra Review

Overfitting / Regularization


Bias-Variance tradeoff, Kernel Methods

[MLAPA] 14.1, 14.2, 14.4;
[ESL] 5.8, 6.3, 6.7
HW3 due
2/15 In-class Midterm HW4 released
2/20 No Class -- Presidents' Day

Support Vector Machines -- Geometric Intuition


Kernel SVM

HW4 due; HW5 released (supplementary file)


[MLAPA] 16.4;
[ESL] 16.1-16.2

Neural Networks (part1, part2)

[MLAPA] 16.5.1-6, 28;
[ESL] 11.3-11.7
HW5 due; HW6 released (supplemental files)

Clustering and Mixture Models

[MLAPA] 11.1-11.5;
[ESL] 14.3.1-9
3/13 No Class (Prof. Talwalkar will hold office hours in BH 4531F) [MLAPA] 12.2; [ESL] 14.5.1

In-class Final

HW6 due