Machine Learning, 15:681, Fall 1997
Professor Tom M. Mitchell
School of Computer Science, Carnegie Mellon University
Machine Learning is concerned with computer programs that
automatically improve their performance through experience. This
course covers the theory and practice of machine learning from a
variety of perspectives. We cover topics such as learning decision
trees, neural network learning, statistical learning methods, genetic
algorithms, Bayesian learning methods, explanationbased learning, and
reinforcement learning. The course covers theoretical concepts such as
inductive bias, the PAC and Mistakebound learning frameworks, minimum
description length principle, and Occam's Razor. Programming
assignments include handson experiments with various learning
algorithms. Typical assignments include neural network learning for
face recognition, and decision tree learning from databases of credit
records.
Time and Place: Tues & Thurs 12:001:20, Porter Hall 125C
Instructor:
Tom Mitchell, Wean
Hall 5309, x82611, Office hours: Wed 3:004:00
Teaching Assistants:
Frank
Dellaert, Smith Hall 212, x86880, Office hours: Thursday 3:305:00
Belinda Thom, Wean Hall 4610,
x83608, Office hours: Tuesday 3:305:00
Course Secretary:
Jean
Harpley, Wean Hall 5313, x83802
Textbook: Machine
Learning, Tom Mitchell, McGraw Hill, 1997.
This course is a combination upperlevel undergraduate and introductory
graduate course.
Ph.D. students in CS may obtain one core credit unit by arranging an
extra course project.
Grading: Will be based on homeworks (35%), midterm (30%), and
final (35%).
Policy on late homework:

Homework is worth full credit at the beginning of class on the due date,

It is worth half credit for the next 48 hours,

It is worth zero credit after that.

You must turn in all but two assignments, even if for zero credit

Free exemption: We will ignore your lowest homework grade for the semester.
Homework assignments (postscript)

Assignment 1 .
Due Sept 11, 1997. Concept learning, sample complexity.

Assignment 2 .
Due Sept 25. Decision tree learning, credit analysis dataset.

Assignment 3.
Due Oct 16. Neural Network learning, face recognition dataset.

Assignment 4.
Due Oct 28. Confidence intervals and Bayesian reasoning.

Assignment 5.
Due Nov 13. Genetic algorithms, Inductive logic programming.

Assignment 6.
Due Dec 2. Naive Bayes for text classification.
Lecture Notes (postscript)

Aug 26, 1997. Overview
and design of a checkers learner. (Chapter 1)

Aug 28. No lecture today.

Sept 2 and 4. Learning
concepts, Version Spaces, inductive bias. (Chapter 2)

Sept 9. Sample complexity,
PAC framework. (Chapt 7, sections 7.2 and 7.3 up to (not including)
7.3.1.)

Sept 11. PAC learning,
VC dimension (remainder of 7.3 except for 7.3.3.2, read 7.4 through
7.4.3, read 7.5 through 7.5.2)

Sept 16. Decision trees
(Chapter 3 through 3.6)

Sept 18. Decision trees
II ( Chapter 3.7 through 3.8)

Sept 23. Neural networks
(Chapter 4)

Sept 25. Neural networks
II (Chapter 4)

Sept 30. Neural nets
III. (Chaper 4)

Oct 2. Review for Midterm
 Oct 7.
Midterm exam. in class, open notes, open book.

Oct 9. Estimation and confidence intervals (Chapter 5). Guest lecture:
Prof. Meyer

Oct 14.
Estimation and confidence intervals II (Chapter 5), and relation to PAC model

Oct 16.
Bayesian learning methods (Chapter 6).
 Oct 21.
Bayesian learning methods II (Chapter 6).

Oct 23.
Bayesian learning methods III (and text classification) (Chapter 6).

Oct 28. Instancebased learning (Chapter 8). Guest lecture:
Frank Dellaert

Oct 30.
Genetic algorithms, Genetic programming (Chapter 9).

Nov 4.
Genetic algorithms II (Chapter 9).

Nov 6.
Learning Sets of Rules, ILP (Chapter 10).

Nov 11.
Explanation based learning (Chapter 11 thru 11.3)

Nov 13.
Combining Inductive and Analytical Learning (Chapter 12 thru 12.3.)

Nov 18.
Bayesian Belief Networks, EM (Chapter 6.11, 6.12)

Nov 20. Hidden Markov Models. Guest lecture: Prof. Rosenfeld

Nov 25.
Reinforcement learning (Chapter 13)
 Dec 2. Reinforcement learning II (Chapter 13), Wrapup: Summary and
Perspectives

Dec 4. No lecture. Optional review session for final.

Dec 9. Final exam. 5:308:30 pm, Wean 7500, open notes, open book.
The course syllabus.
Note to people outside CMU
Feel free to use the slides and materials available online here. Please
email Tom.Mitchell@cmu.edu with
any corrections or improvements.
See also the Fall
1996 version of this course , cotaught with Prof. Avrim Blum.