.po 1i
.ll 6.5i
.sz 16
.ce
\fBCourse Syllabus for CS 395T: Machine Learning\fP
.sp 2
.sh 1 Introduction
.pp
Definitions, goals, and history of Machine Learning.  Taxonomies of methods and
research paradigms.  Knowledge-level vs. symbol-level learning.  Representation
of acquired concepts.  Theoretical and empirical evaluation of learning
algorithms.
.sp .5
Readings:
.sp .5
"Introduction", pp. 1-11.
.sp .5
"Machine Learning as an Experimental Science," pp. 38-43.
.sh 1 "Inductive Concept Acquisition from Examples"
.pp
Acquiring general concept descriptions by examining the similarities and
differences between teacher-supplied examples and counter-examples. Learning as
search through a space of concept descriptions.  Learning from only positive
examples or from positive and negative examples.  Incremental vs.  batch
learning.  The importance of inductive bias, Occam's Razor.  The single
representation trick.  The need for constructive induction.  The problem of
noisy data.
.sp .5 
Readings:
.sp .5
"Inductive Learning from Preclassified Training Examples", pp. 45-56.
.sp .5
"A Theory and Methodology of Inductive Inference," pp. 70-84 only.
.sh 2 "Version-Space Algorithm"
.pp
Alternative searching techniques. Basic version-space algorithm.  Computational
complexity.  Representational restrictions.
.sp .5 
Reading:
.sp .5
"Generalization as Search," pp. 96-107.
.sh 2 "ID3"
.pp
Decision trees as concept classifiers. Basic ID3 algorithm.  Computational
complexity.  Representational restrictions.  Handling noise and missing feature
values in ID3.  Incremental versions ID4 and ID5.  Constructive induction and
the FRINGE algorithm.
.sp .5 
Reading:
.sp .5
"Induction of Decision Trees," pp. 57-69.
.sh 2 "AQ and INDUCE"
.pp
Handling disjunction.  Tradeoff between optimality and complexity.
Basic AQ algorithm.
.sp .5 
Reading:
.sp .5
"A Theory and Methodology of Inductive Inference," pp. 85-95.
.sh 1 "Formal Learnability Models"
.pp
Formal definitions of learnability. Gold's results on learnability in the
limit.  Valiant's results on probabilistic polynomial-time learnability (PAC:
Probably Approximately Correct).  Using the size of the hypothesis space to
quantify bias.  Vapnik-Chervonenkis dimension as a measure of a hypothesis
space.
.sp .5 
Reading:
.sp .5
"Quantifying Inductive Bias: AI Learning Algorithms and Valiant's
Learning Framework," pp. 205-227.
.sh 1 "Learning by Observation and Discovery"
.pp
Learning without a teacher. Conceptual Clustering in CLUSTER/2, UNIMEM, and
COBWEB.  Clustering and prediction.  Incremental vs. batch clustering.
Discovery as heuristic search. Mathematical discovery in AM. Scientific
discovery in BACON.
.sp .5 
Readings:
.sp .5
"Unsupervised Concept Learning and Discovery," pp. 263-266, 337-340.
.sp .5
"Knowledge Acquisition Via Incremental Conceptual Clustering," pp. 267-283.
.sp .5
"The Ubiquity of Discovery," pp. 341-355.
.sp .5
"Heuristics for Empirical Discovery," pp. 356-372.
.sh 1 "Explanation-Based Learning"
.pp
Using existing domain knowledge to explain and generalize examples of concepts
and plans.  Learning for problem solving and understanding.  Generalization
algorithms, chunking, and macro-operators.  Systems: STRIPS, SOAR, EBG, &
EGGS. Effect on performance. Learning apprentice systems.  Operationality vs.
generality.
.sp .5 
Readings:
.sp .5
"Improving the Efficiency of a Problem Solver", pp. 429-434, 503-509.
.sp .5
"Explanation-Based Generalization: A Unifying View," pp. 435-451.
.sp .5
"Explanation-Based Learning: An Alternative View" pp. 452-467.
.sp .5
"Learning by Experimentation: Acquiring and Refining Problem Solving Heuristics," 510-523.
.sp .5
"Explanatory/Inductive Hybrids," pp. 747-753.
.sh 1 "Connectionist Learning"
.pp
Learning by adjusting connection strengths between large numbers of simple
processors operating in parallel.  Neurological motivation. Perceptron
learning.  The problem of linear separability.  Multilayer networks and hidden
units.  Backpropagation.  Local vs. distributed representations.
.sp .5 
Readings:
.sp .5
"Perceptrons: Learning," pp.228-241.
.sp .5
"Learning Internal Representations by Error Propagation," pp. 115-137.
.sp .5
"An Experimental Comparison of Symbolic and Connectionist Learning Algorithms,"
pp. 171-176.
.sh 1 " Exemplars, Case-Based Reasoning and  Analogy"
.pp
Exemplars (i.e. prototypical specific instances) as a representation of
concepts. Protos exemplar-based system.  Solving problems by adapting solutions
to previous specific past cases.  Analogical problem solving and learning.
Analogical access, mapping, and transfer.  Structure mapping theory of analogy
and the SME system.
.sp .5 
Readings:
.sp .5
"Learning Representative Exemplars of Concepts: An Initial Case Study," 
pp. 108-114.
.sp .5
"Case-based Approaches," pp. 684-685 
.sp .5
"Knowledge Acquisition and Heuristic Classification in Weak-Theory Domains,"
pp. 710-746.
.sp .5
"Analogical Approaches," pp. 597-600.
.sp .5
"Mechanisms of Analogical Learning," pp. 601-622.
.sh 1 "Genetic Algorithms"
.pp
Model of learning based on analogy to biological evolution and genetics.
Genetic operators: cross-over, mutation, and inversion.  Genetic Learning
Algorithm. Classifier systems, credit assignment, and the bucket brigade
algorithm.  Relation to connectionist learning.
.sp .5 
Reading:
.sp .5
"Classifier Systems and Genetic Algorithms," pp. 404-427

