### 15-854 Machine Learning Theory, Spring 1998

### Course Information

**Lectures:** Monday and Wednesday 1:30--2:50, Wean 4615A
**Instructor:** Avrim Blum
(Wean 4107, avrim@cs.cmu.edu, x8-6452).

**Office Hours:** Stop by or make an appointment.

**Credits:** 12 Units, 1 CU

** Course Description: **
This course will focus on theoretical aspects of machine learning. We
will examine questions such as: What kinds of guarantees can one prove
about learning algorithms? What are good algorithms for achieving
certain types of goals? Can we devise models that are both amenable
to mathematical analysis and make sense empirically? What can we say
about the inherent ease or difficulty of learning problems?
Addressing these questions will require pulling in notions and ideas
from statistics, complexity theory, cryptography, game theory, and
empirical machine learning research.

**Evaluation and Responsibilities:**
Grading will be based on 5 or 6 homework assignments, a take-home final, and
class participation. As part
of class participation, students will each give one presentation on a
topic chosen in consultation with the instructor. Students interested
in performing an experimental project based on ideas
discussed in class may be able to do so in place of some of the formal
requirements. Because this course has no TA, students from time to
time will also be asked to help with the grading of assignments.

**General structure of the course:**
We will use roughly 2/3 of the lectures to cover "core" topics in this area,
and then will diverge in the remaining 1/3 based on student interest.
Here is a rough outline of the "core" portion (some bullets will require
more than one lecture):

- Overview, consistency model
- Mistake-bound model: basic algorithms and transformations
- the Weighted-Majority Algorithm
- the Winnow Algorithm and the Infinite-Attribute model
- PAC model: basic results, relations to MB, consistency
- Occam's razor: when can we be confident about our predictions?
- VC-dimension, uniform convergence
- Weak-learning vs. Strong-learning
- Classification noise and the Statistical-Query model
- Fourier analysis
- Algorithmic hardness results, relations to cryptography
- Active learning of finite state environments
- Active learning of decision trees

**Text:**
Kearns and Vazirani, "An introduction to computational
learning theory" plus papers and notes for topics not in the book. (Roughly
half of the topics are in the book)