.po 1i
.ll 6.5i
.nr pp 11
.nr sp 11
.sz 16
.ce
\fBCourse Syllabus for CS 395T: Machine Learning\fP
.sz 11
.sp 2
.uh Abbreviations
.nr ii 1i
.ip ML1
\fIMachine Learning: An Artificial Intelligence Approach\fP, Michalski,
Carbonell, & Mitchell, ed., Tioga Publishing Co., Palo Alto, CA, 1983.
.ip  "AI Handbook:"
\fIThe Handbook of Artificial Intelligence Vol. III\fP, Cohen & Feigenbaum,
ed., William Kaufman Inc., Los Altos, CA, 1982.
.sh 1 Introduction
.pp
Definitions, goals, and history of Machine Learning.  Taxonomies of methods and
research paradigms.  Knowledge-level vs. symbol-level learning.  Representation
of acquired concepts.  Theoretical and empirical evaluation of learning
algorithms.
.sp .5
References: 
.sp .5
AI Handbook, Chapter XIV, "Learning and Inductive Inference," Section A,
"Overview," pp. 325-334.
.sp .5
Carbonell, Michalski, & Mitchell, "An Overview of Machine Learning," ML1,
Chapter 1, pp. 3-16.
.sp .5
Carbonell, J.G., "Paradigms for Machine Learning," \fIArtificial
Intelligence\fP, 40, 1, 1989, pp. 1-9.
.sh 1 "Inductive Concept Acquisition from Examples"
.pp
Acquiring general concept descriptions by examining the similarities and
differences between teacher-supplied examples and counter-examples. Learning as
search through a space of concept descriptions.  Learning from only positive
examples or from positive and negative examples.  Incremental vs.  batch
learning.  The importance of inductive bias, Occam's Razor.  The single
representation trick.  The need for constructive induction.  The problem of
noisy data.
.sp .5 
Reference:
.sp .5
AI Handbook, Chapter XIV, Section D1, "Learning from Examples: Issues,"
pp. 360-372. 
.sp .5
Michalski, R.S., "A Theory and Methodology of Inductive Inference,"
ML1, pp. 83-112.
.sh 2 "Version-Space Algorithm"
.pp
Alternative searching techniques. Basic version-space algorithm.  Computational
complexity.  Representational restrictions.
.sp .5 
Reference:  
.sp .5
Mitchell, T.M., "Generalization as Search," \fIArtificial Intelligence\fP,
18, 2, 1982, pp. 203-226.
.sh 2 "ID3"
.pp
Decision trees as concept classifiers. Basic ID3 algorithm.  Computational
complexity.  Representational restrictions.  Handling noise and missing feature
values in ID3.  Incremental versions ID4 and ID5.  Constructive induction and
the FRINGE algorithm.
.sp .5 
Reference:
.sp .5
Quinlan, J.R., "Induction of Decision Trees," \fIMachine Learning\fP, 1, 1,
1986, pp. 81-106.
.sh 2 "AQ and INDUCE"
.pp
Handling disjunction.  Tradeoff between optimality and complexity.
Basic AQ algorithm.
.sp .5 
References:
.sp .5
Michalski, R.S., "A Theory and Methodology of Inductive Inference,"
ML1, pp. 112-130.
.sp .5
AI Handbook, "Disjunctive Concepts," pp.397-400;  "D4a. AQ11," pp.423-427;
"D3c. Concept Learning by Generating and Testing Plausible Hypotheses," pp.
411-415.
.sh 1 "Formal Learnability Models"
.pp
Formal definitions of learnability. Gold's results on learnability in the
limit.  Valiant's results on probabilistic polynomial-time learnability (PAC:
Probably Approximately Correct).  Using the size of the hypothesis space to
quantify bias.  Vapnik-Chervonenkis dimension as a measure of a hypothesis
space.
.sp .5 
References:
.sp .5
Haussler, D., "Quantifying Inductive Bias: AI Learning Algorithms and Valiant's
Learning Framework," \fIArtificial Intelligence\fP, 36, 2, 1988.
.sh 1 "Learning by Observation and Discovery"
.pp
Learning without a teacher. Conceptual Clustering in CLUSTER/2, UNIMEM, and
COBWEB.  Clustering and prediction.  Incremental vs. batch clustering.
Discovery as heuristic search. Scientific discovery in BACON.  Mathematical
discovery in AM.
.sp .5 
References:
.sp .5
Michalski, R.S. and Stepp, R. E., "Learning from Observation: Conceptual
Clustering," ML1, pp. 331-363.
.sp .5
Fisher, D., "Knowledge Acquisition Via Incremental Conceptual Clustering,"
\fIMachine Learning\fP 2, 2, 1987, pp. 139-172.
.sp .5
Langley, P. "Data-Driven Discovery of Physical Laws," \fICognitive Science\fP
5, 1981, pp. 31-54.
.sp .5
AI Handbook, "D4c. AM" pp. 438-451.
.sh 1 "Explanation-Based Learning"
.pp
Using existing domain knowledge to explain and generalize examples of concepts
and plans.  Learning for problem solving and understanding.  Generalization
algorithms, chunking, and macro-operators.  Systems: STRIPS, SOAR, EBG, &
EGGS. Effect on performance. Learning apprentice systems.  Operationality vs.
generality.
.sp .5 
References:
.sp .5
Mitchell, T.M., Keller, R.M., & Kedar-Cabelli, S., "Explanation-Based
Generalization: A Unifying View," \fIMachine Learning\fP 1, 1, 1986, pp.
47-80.
.sp .5
Mooney, R.J., "Explanation Generalization in Eggs" in \fIInvestigating
Explanation-Based Learning\fP, G.F. DeJong (ed.), Kluwer Academic Publishers,
Boston, MA, forthcoming.
.sh 1 "Connectionism"
.pp
Learning by adjusting connection strengths between large numbers of simple
processors operating in parallel.  Neurological motivation. Perceptron
learning.  The problem of linear separability.  Multilayer networks and hidden
units.  Backpropagation.  Local vs. distributed representations.
.sp .5 
References:
.sp .5
Rumelhart, D.E., McClelland, J. L., and the PDP Research Group, 
\fIParallel Distributed Processing: Explorations in the Microstructure of
Cognition, Volume 1: Foundations\fP, MIT Press, Cambridge, MA. 1986.
Chapter 2, "A General Framework for Parallel Distributed Processing,"
by Rumelhart, Hinton, and McClelland, pp. 45-76; and Chapter 8, "Learning
Internal Representations by Error Propagation," by Rumelhart, Hinton, and
Williams, pp. 318-362.
.sp .5
Mooney, R.J., J.W. Shavlik, G. Towell, and A. Gove, "An Experimental Comparison
of Symbolic and Connectionist Learning Algorithms," \fIProceedings of the
Eleventh International Joint Conference on Artificial Intelligence\fP, Detroit,
MI, August 1989.
.sh 1 " Exemplars, Case-Based Reasoning and  Analogy"
.pp
Exemplars (i.e. prototypical specific instances) as a representation of
concepts. Protos exemplar-based system.  Solving problems by adapting solutions
to previous specific past cases.  Analogical problem solving and learning.
Analogical access, mapping, and transfer.  Structure mapping theory of analogy
and the SME system.
.sp .5 
References:
.sp .5
Kibler, D. and Aha, D. W., "Learning Representative Exemplars of Concepts:
An Initial Case Study," \fIProceedings of the Fourth International Workshop
on Machine Learning\fP, Irvine, CA, Morgan Kaufman Pub., 1987, pp. 24-30.
.sp .5
Porter, B.W., Bariess, E.R., and Holte, R.C., "Knowledge Acquisition and
Heuristic Classification in Weak-Theory Domains," \fIArtificial Intellgence\fP,
to appear.
.sp .5
Gentner, D. "Mechanisms of Analogical Learning," \fISimilarity and Analogical
Reasoning\fP, S. Voniadou and A. Ortony (Eds.), Cambridge University Press,
London, 1989, pp. 199-241.
.sh 1 "Genetic Algorithms"
.pp
Model of learning based on analogy to biological evolution and genetics.
Genetic operators: cross-over, mutation, and inversion.  Genetic Learning
Algorithm. Classifier systems, credit assignment, and the bucket brigade
algorithm.  Relation to connectionist learning.
.sp .5 
References:
.sp .5
Booker, L.B., Goldberg, D.E., and Holland, J.H., "Classifier Systems and
Genetic Algorithms," \fIArtificial Intelligence\fP, 40, 1-3, 1989, pp. 235-282.

