Overview of Probability& Statistics

- Overview of machine learning and its applications
- Probability & Statistics overview
- Random variables, expectations, distributions, Bayes rule, prior, posterior

Mon., Aug. 24 (Geoff)

- Lecture:Overview of machine learning and applications [Slides] [Annotated]
- Lecture:Overview of probability and statistics [Annotated1]
- In the annotated version, some slides are cutoff. Here we include a version of the same lecture, where there is type setted notes below each slide. [Annotated2]
- Readings: Bishop pages 1-4 and section 1.2

Wed., Aug. 26 (Geoff):

- Lecture:Overview of probability and statistics [Slides] [Annotated]
- Readings: Bishop sections 2.0-2.3

Graphical Models

Mon., Aug.31 (Geoff)

- Lecture: Introduction to graphical models [Slides] [Annotated]

Wed., Sep.2 (Geoff)

- Lecture: Introduction to graphical models [Slides] [Annotated]

Wed., Sep.9 (Geoff)

- Lecture: Graphical models continued. [Slides] [Annotated]
- Code to plot the two dimensional continous probability function that was shown in the lecture: code
- Readings: Bishop pages 359-362 (sec 8-8.1),365-369 (sec 8.1.2-8.1.3),372-383 (sec 8.2),393-402 (sec 8.4-8.4.3),optional: sec 8.3,8.4.4

Density Estimation

Wed., Sep.9 (Miro)

- Lecture: Density estimation [Slides]
- Readings: Bishop sections 1.2.3, 2.0, 2.1

Mon., Sep.14 (Miro)

- Lecture: Density estimation [Slides]
- There is a correction in slides the corrected version is [Annotated]

Regression

Wed., Sep.16 (Miro)

- Lecture: Gaussians and Regression [Annotated]
- Readings: Bishop pages 78-79 (sec 1.2.4, 2.3.0), 97-98 (sec 2.3.4, 2.3.6),4-9 (sec 1.1), 138-140 (sec 3.1.0).

Mon., Sep. 21 (Miro)

- Lecture: Regression and model selection [Annotated]
- Readings: Bishop pages 10-12 (sec 1.1) and sections 1.2.5, 3.1.1, 3.1.2, 3.1.4, 3.2

Classification

Wed., Sep. 23 (Miro)- Lecture: Model selection and Naive Bayes [Annotated]
- Readings: Bishop sections 1.3, 1.5 Tom Mitchell's Chapter on Naive Bayes (Sections 1 and 2) can be accessed here

Mon., Sep. 28 (Geoff)

- Lecture: Naive Bayes and Multivariate Distributions [Slides] [Annotated]
- Readings: Bishop sections 4-4.1.2, 4.1.4, 4.1.6, 4.2-4.2.3, 4.3-4.3.2, 4.3.4 Code used to plot the figures in the lecture:
- ellipse.m , drawhalfspace.m , plot10.m

Wed., Sep. 30 (Geoff)

- Lecture: Logistic regression [Slides] [Annotated]
- Code for the plots 11-plots.m
- Here is a page that Geoff's prepared. It works through an example of fitting a logistic model with the iteratively-reweighted least squares IRLS) algorithm.
- Readings: Rest of Tom Mitchell's Chapter on Naive Bayes, Logistic Regression & Generative vs Discriminative (optional) can be accessed here

Approximate Inference

Mon., Oct. 5 (Geoff)

- Lecture:MCMC Sampling [Slides] [Annotated] Code used to plot the figures in the lecture: 12-plots.m
- Readings: Bishop sections Ch 11's intro (p523-525), 11.1.4

Wed., Oct. 7 (Geoff)

- Lecture: MCMC Sampling [Slides] [Annotated] Videos shown in the class: stationary-dist.mov , particle-filter.mov , wander.mov
- Decription of the videos: videos.txt
- Readings: Bishop sections 11.2-11.3

Latent Variables Analysis

Mon., Oct. 12 (Geoff)

- Lecture: Sampling continued and Principal Component Analysis [Slides] [Annotated]
- Readings: Bishop pages 559-577.

Wed., Oct. 14 (Geoff)

- Lecture: Latent variable models continued [Slides] [Annnotated
- Readings: Hastie-Tibshirani-Friedman The Elements of Statistical Learning 2nd edition 14.5 (PCA, principal curves) and 14.10 (PageRank), accesible here.

Mon., Oct. 19 (Geoff + Miro)

- Lecture: Bayesian PCA [Slides] [Annnotated]
- Lecture: k-Nearest Neighbor Decision Trees [Slides]
- Readings: k-Nearest Neighbor Classifiers: Bishop Figures 2.27, 2.28,Hastie-Tibshirani-Friedman Section 2.3.2.

Decision Trees

Wed., Oct. 21 (Miro)

- Lecture: Decision Trees [Slides]
- Readings: Bishop Sections 1.6, 14.4

Learning Theory

Mon., Oct. 24 (Miro)

- Lecture: PAC learning and Occam's razor [Notes]
- Readings:
- PAC learning & Occam's Razor: Sections 1&2 of these notes.
- Inconsistent hypothesis model: Sections 2&3 of: these notes. Sections 1&3 of: these notes.

- Lecture: VC dimension and online learning [Notes]
- Readings :
- VC dimension: Section 4 of: these notes.
- Sections 1&3 of: these notes.
- Online learning: Sections 2-5 of these notes.

Wed., Oct. 28 (Miro)

Online learning and Ensemble Learning

Wed., Nov. 4 (Miro)

- Lecture:Online learning and ensemble learning [Slides]
- Readings:
- Randomized weighted majority and perceptron (optional reading): Notes
- Ensemble learning: Paper
- Bagging & random forests (optional reading): Section 8.7 and beginning of Section 5 (pages 587-592) of Hastie, Tibshirani, Friendman, 2nd ed: Book

Mon., Nov. 9 (Miro)

- Lecture:Boosting [Slides] [Annnotated]
- Readings: Reading: Sections 1-4 and 8-10 of these notes

Support Vector Machines

Wed., Nov. 11 (Miro)

Mon., Nov. 16 (Miro)

- Lecture: Support Vector Machines continued. [Slides]

Wed., Nov 18 (Miro)

- Lecture: Support Vector Machines continued. [Slides] [Annotated]

Clustering

Mon., Nov 23 (Geoff)

- Lecture: Clustering [Slides] [Annotated] Code used to plot the figures in the lecture: 25_clustering.m
- Readings: Bishop 9.1 9.2.
- Lecture: Clustering [Slides] [Annotated] Code used to plot the figures in the lecture: em_demo.m

Mon., Nov 30 (Geoff)