- School of Computer Science, Carnegie Mellon University
- Wean 3412, Tues & Thurs 12:00-1:20

Instructors:

Teaching Assistant:

Course textbook:
*Machine Learning,* Tom Mitchell, McGraw Hill

Copies of the textbook can be picked up in Jean Harpley's office: Wean Hall 5313.

Machine Learning is concerned with computer programs that automatically improve their performance through experience. Machine Learning methods have been applied to problems such as learning to drive an autonomous vehicle, learning to recognize human speech, learning to detect credit card fraud, and learning strategies for game playing. This course covers the primary approaches to machine learning from a variety of fields, including inductive inference of decision trees, neural network learning, statistical learning methods, genetic algorithms, bayesian methods, explanation-based learning, and reinforcement learning. The course will also cover theoretical concepts such as inductive bias, the PAC and Mistake-Bound learning framework, Occam's Razor, uniform convergence, models of noise, and Fourier analysis. Programming assignments include experimenting with various learning problems and algorithms. This course is a combination upper-level undergraduate and introductory graduate course. CS Ph.D. students can obtain one core credit unit by arrangement with the instructor.

Here is a **Course syllabus**.

- General Course Information
- Homework 1. Due Sept 10, 1996. Average grade: about 14/20. Solutions.
- Homework 2. Due Sept 17, 1996. Average grade: about 15/20.
- Homework 3. Due Sept 26, 1996. Average grade: about 17/20.
- Homework 4. Due Oct 3, 1996. Average grade: about 15/20. Solutions.
- Homework 5. Part I due Oct 22, Part II due Oct 29.
- Homework 6. Due November 14. Solutions.
- Homework 7. Due December 5. **NOTE: DELETE QUESTION 3. IT IS BUGGY! ***

- Aug 27. Overview and design of a checkers learner. (Read Chapter 1)
- Aug 29. Concept learning, version spaces, inductive bias. (Read Chapter 2)
- Sept 3. Theoretical frameworks: consistency and PAC models. (Read Chapter 7.1-7.3)
- Sept 5. PAC model contd, Decision lists, Occam's razor.
- Sept 10. Decision tree learning (Read Chapter 3)
- Sept 12. Subtleties of decision tree learning. (Read Chapter 3)
- Sept 17. Evaluating hypotheses. (Read Chapter 5.1-5.4)
- Sept 19. More on evaluating hypotheses (Rest of Chapter 5), online learning, weighted majority algorithm.
- Sept 24. Weighted majority algorithm and applications. (Read Chapter 7.5)
- Sept 26. More on on-line learning, Winnow algorithm.
- Oct 1. Neural networks (Chapter 4.1-4.5.1).
- Oct 3. Neural networks (Chapter 4.5.2-4.6.4 (skip 4.5.3)).
- Oct 8.
**MIDTERM EXAM**(in class, open book) Grading info. Solutions. - Oct 10. Neural nets, C[m], "effective degrees of freedom", VC-dimension. (Chapter 7.4)
- Oct 15. Bayesian learning (Chapter 6)
- Oct 17. More Bayes, Max Likelihood and sum of squared error, Minimum Description Length Principle (Chapter 6)
- Oct 22. Finish Bayes: Bayes optimal, relation to WM, using Naive Bayes to classify text (Chapter 6).
- Oct 24. Boosting: Theory and applications.
- Oct 29. Class presentations: neural nets and face recognition (assignmt 5)
- Oct 31. Nearest neighbor (Chapter 8)
- Nov 5. Genetic algorithms, Genetic programming (Chapter 9)
- Nov 7. Schema Theorem, Sequential Covering Algorithms (Chapt 9, 10)
- Nov 12. Inductive Logic Programming, FOIL, CIGOL (Chapter 10).
- Nov 14. Active learning, learning finite state devices.
- Nov 19. Explanation Based Learning (Chapter 11).
- Nov 21. Combining inductive and analytical learning; KBANN (Chapter 12).
- Nov 26. Combining inductive and analytical learning; EBNN, FOCL (Chapter 12), and The EM (Estimation/Maximization) Algorithm.
- Dec 3. Hidden Markov Models.
- Dec 9.
**FINAL EXAM**8:30-11:30, MM 103

See also Fall 1995 version of this course, including midterm and final exam.