.po 1i
.ll 6.5i
.nr ps 11
.nr pp 11
.ce 10
\fB\s16Homework 6: PERCEPTRON and PDP\fP\s0
.sp
.sz 14
CS 395T: Machine Learning
.sp .5
Due: Thursday, April 21
.sp
.ce 0
.pp
Common Lisp implementations of the perceptron and error backpropagation
(generalized delta rule) connectionist learning algorithms are in the files
PERCEPTRON and PDP respectively. Both are commented and contain the necessary
information for their use. Both can be run on learning from examples data in
FIGURE-DOMAIN, WEATHER-DOMAIN, SOYBEAN-RDOMAIN, and SOYBEAN-DOMAIN.  Some
additional examples for PDP are in the file PDP-DATA. (Time constraints
prevent running PDP on soybean data but try it if you like). There is an XOR
example in the file PERCEPTRON and PDP-DATA.
.pp
Currently, PERCEPTRON will only terminate if the inputs are linearly
separable. Your task is to change the system so that after each
iteration through the examples it checks to see if the current
weights and threshold are the same as some past state of the system
(EQUALP can be used to test array equivalence).  Since the system
always processes the examples in the same order during each iteration,
once the the system repeats the same weights and threshold, future 
behavior will repeat past behavior and the system is caught in an infinite
loop.  In this case, exit the system and output that the inputs are
not linearly separable.  (I am not absolutely certain that the inverse holds
(i.e. that if the inputs are not linearly separable then the system will
eventually repeat a past state); however, I believe this is what
Minsky and Papert's "Perceptron Cycling Theorem" indicates.)
.pp
Backpropogation is a very time consuming algorithm which is not guaranteed to
converge although it can learn from examples which are not linearly separable.
Perceptron learning either eventually converges or goes into an infinite loop
and in practice runs fairly quick even on large examples; however, it cannot
solve problems which are not linearly separable.  After adding loop detection
to PERCEPTRON, combine it  with PDP in the following way, first try
running PRECEPTRON on an example and if it doesn't converge then call
PDP as a last resort.  You will have to use to function PDP-EXAMPLE-FORM
to convert a problem to the appropriate representation for PDP.
Test the final system on the XOR problem and hand hand in your commented
code and a trace for the XOR example. Include a paragraph or two
discussing whether or not you think this combination is a good idea.


