Basics

• What is learning?
• Version spaces
• Sample complexity
• Training set/Test set split
• Point estimation
• Loss functions
• MLE
• Bayesian
• MAP

Mon., Sep. 10:

[Top]

Linear Models

• Linear regression [Applet]
http://www.mste.uiuc.edu/users/exner/java.f/leastsquares/
• Overfitting
• Bayes optimal classifier
• Naive Bayes [Applet]
http://www.cs.technion.ac.il/~rani/LocBoost/
• Logistic regression [Applet]
• Discriminative v.Generative models [Applet]

Wed., Sep. 12:

• Lecture: Gaussians, Linear Regression, Bias-Variance Tradeoff, Overfitting, What's ML revisited. [Slides] [Annotated]
• Readings: Bishop 1.1 to 1.4, Bishop 3.1, 3.1.1, 3.1.4, 3.1.5, 3.2, 3.3, 3.3.1, 3.3.2
• Completely Optional: Joey's quickly written notes on the matrix MLE for regression. [PDF] [Mathematica6 Notebook] If there are any typos or mistakes please let me know .

Mon., Sep 17:

Wed., Sep 19:

Mon., Sep 24:

[Top]

Non-linear models and Model selection (4 Lectures)

• Decision trees [Applet]
• Overfitting, again
• Regularization
• MDL
• Cross-validation
• Instance-based learning [Applet] from www.site.uottawa.ca/~gcaron/applets.htm
• K-nearest neighbors
• Kernels
• Neural nets [CMU Course] from www.cs.cmu.edu/afs/cs/academic/class/15782-s04/ [Applet] from http://neuron.eng.wayne.edu/bpFunctionApprox/bpFunctionApprox.html

Wed., Sep. 26:

Mon., Oct. 1:

Wed., Oct. 3:

Mon., Oct. 8:

• Lecture: Neural Nets [Slides]
• Readings: (Bishop 5.1) Feed-forward Network Functions
• (Bishop 5.2) Network Training
• (Bishop 5.3) Error Backpropagation

[Top]

Margin-based approaches (3 Lectures)

• SVMs [Applets] from www.site.uottawa.ca/~gcaron/applets.htm
• Kernel trick

Wed., Oct. 10:

Mon., Oct. 15:

Wed., Oct. 17:

[Top]

Learning Theory (2 Lectures)

• Sample complexity
• PAC learning [Applets]
www.site.uottawa.ca/~gcaron/applets.htm
• Error bounds
• VC-dimension
• Margin-based bounds
• Large-deviation bounds
• Hoeffding's inequality, Chernoff bound
• Mistake bounds
• No Free Lunch theorem

Wed., Oct. 24:

[Top]

Midterm

Thu., Oct 25 5-6:30pm
location: MM A14

[Top]

Structured Models (4 Lectures)

• HMMs
• Forwards-Backwards
• Viterbi
• Supervised learning
• Graphical Models

Mon., Oct. 29:

Wed., Oct. 31:

Mon., Nov. 5:

Wed., Nov. 7:

[Top]

Unsupervised and semi-supervised learning (4 Lectures)

• K-means (Applet: K-means)
• Expectation Maximization (EM)
• Combining labeled and unlabeled data
• EM
• reweighting labeled data
• Co-training
• unlabeled data and model selection
• Dimensionality reduction (PCA, SVD) Applet: PCA
• Feature selection

Mon., Nov. 12:

• Lecture: BNs Structure learning, Clustering - K-means [Slides] [Annotated]
• Readings: (Bishop 9.1, 9.2) - K-means, Mixtures of Gaussian

Wed., Nov. 14:

• Guest Lecture: Online Learning (Avrim Blum) [Slides]

Wed., Nov. 21:

• NO CLASS: Thanksgiving

Mon., Nov. 26:

[Top]

Learning to make decisions (2 Lectures)

• Markov decision processes
• Reinforcement learning

Wed., Nov. 28:

Special date/time: Thursday, Nov. 29th, 5-6:20pm in Wean 7500:

Fri., Nov. 30:

## Project Poster Session

2-5pm, Newell-Simon Hall Atrium

[Top]

## Final Exam

Tuesday, Dec. 11, 5:30-8:30PM

Location TBA

[Top]

## Project Paper Due

2pm, Friday, Dec. 14

[Top]