                     README file for Hyperplane Animator
				Version 1.01
			Lori Pratt and Steve Nicodemus
	       Department of Mathematical and Computer Sciences
			    Colorado School of Mines

			  Previous version of animator
			    designed and written by:
			Paul Hoeper, Paul Munro, Abby Dy
		  Mick Noordewier, Harry Walker, Matt Dismukes
				  Vincent Sgro


			
                     -----------------------------------
			     October 10, 1993


-- OVERVIEW --
This directory contains code and sample data files for a Hyperplane
Animator. These files are currently being maintained by Lori Pratt and
Steve Nicodemus at the Colorado School of Mines.  The animator runs
under X, and makes use of color, if available.  It is based on the idea
that neural network weights can be interpreted as hyperplanes in
feature space or in hidden unit activation space (for hidden-to-output
weights), and that important aspects of neural network learning
dynamics can be understood by animating these hyperplanes as they move
into position during learning.  We have used it to help with the
research reported in [Ramachandran & Pratt, 1992] and [Pratt, 1993].


Contents:
 1. What is the Hyperplane Animator?
 2. Why use it?
 3. General Description.
 4. How to install the Animator.
 5. Running the Animator, overview.
 6. Quirks.
 Appendix:  Input file format
 References


1. What is the Hyperplane Animator?

The Hyperplane Animator is a program that allows easy graphical display
of Back-Propagation training data and weights in a Back-Propagation
neural network.

Back-Propagation neural networks consist of processing nodes
interconnected by adjustable, or ``weighted'' connections.  Neural
network learning consists of adjusting weights in response to a set of
training data.  The weights w1,w2,...wn on the connections into any one
node can be viewed as the coeffiecients in the equation of an
(n-1)-dimensional plane.  Each non-input node in the neural net is thus
associated with its own plane.  These hyperplanes are graphically
portrayed by the hyperplane animator.  On the same diagram it also
shows the training data.


2. Why use it?
As learning progresses and the weights in a neural net alter, the
positions of the hyperplanes mentioned above should move around
eventually comming to rest in places that neatly divide the data points
into partitions of all one type or another.

The ability to see these hyperplanes, and how they move, can yield valuable 
insights into how neural networks learn.  We have used it to help with the 
research reported  in [Ramachandran & Pratt, 1992] and [Pratt, 1993].


3.  General Description.
The hyperplane animator requires a log from a network training session
as input.  In this file each weight and bias must have been recorded at
each epoch.  Thus the program displays not real-time training, but
instead shows animation of a previously trained network.  The user must
first use <FILE> from the menu to open a file previously created with
the correct format.  From there use any of the control buttons to start
or stop the animation, to control how fast/slow the animation goes, and
to adjust other display parameters such as color and data point size.


3. How to install the Animator
The Animator was developed using the Motif toolkit on an IBM RS6000
with X-Windows.  It appears to be stable on this platform, and has not
been compiled on other platforms.  However, Dec5000 and SGI workstations
have been succesfully used as graphics servers for the animator.

To install the Animator, you will need a machine which has X-Windows,
and the Motif libraries.

  1. copy the file animator.tar.Z to your machine via ftp as follows:

     ftp mines.colorado.edu (138.67.1.3)
     Name: anonymous
     Password: (your ID)
     ftp> cd pub/software/hyperplane-animator
     ftp> binary
     ftp> get hyperplane-animator.tar
     ftp> quit

  2. Extract files from hyperplane-animator.tar with:
     tar -xvf hyperplane-animator.tar
  
  3. To compile the hyperplane animator, cd to the Src directory, and
     use the following compile command: 

	cc -o animator animator.c coloredit.c credits.c prefs.c  \
              colorsave.c printfnct.c fileread.c prefchanges.c \
	      hideshow.c -lXm -lXt -lX11

After successfully compiling the hyperplane animator, you can use any
of demo networks that we've included.  See ``Walk-Through Example''  in
the included document UserManual.txt.

4. Running the Animator.

The hyperplane animator requires a log from a network training session
as input.  In this file each weight and bias must have been recorded at
each epoch.  Thus the program displays not the real-time training of a
network, but is instead an animation of a previously trained network.
The program operates by first asking for an input file.  Then it
creates the display window.  From here the user can use any of the
control buttons described below to start or stop the animation, to
control how fast/slow the animation goes, and to adjust other display
parameters like color and data point size.

5. Quirks.
Most popup windows have a CLOSE button.  The rule is:  If a popup
window has a CLOSE button, you MUST USE THAT BUTTON to close the
popup.  (For windows with no close button, use the Motif window manager
close feature, ie click mouse on minus sign in upper left corner of
window.) Unfortunately, it's possible to close a window without using
the CLOSE button, and nothing un-toward will happen; BUT, when you
attempt to open that window again the app will bomb.

See the UserManual for other tips relating to Color choices.

If you find bugs, or have any suggestions about the Hyperplane
Animator, please write:
	lpratt@mines.colorado.edu       (Lori Pratt, assistant professor, 
					 Department of Mathematical and
					 Computer Sciences, 
					 Colorado School of Mines)


Appendix:  File Format 

The hyperplane animator takes as input a data file representing the
results of neural network learning at each epoch.  Its file format is
described below; you'll need to modify whatever network training
program you use in order to accomodate this.  We use the software
distributed with:  [Mcclelland & Rumelhart, 1988] for our simulations,
and have developed shell scripts and conversion code for allowing this
program to generate the proper input format for the hyperplane
animator.  Contact us if you are interested in this code.

For an animation sequence involving:
   H as the number of hyperplanes (i.e. H nodes on the first hidden layer),
   P training data points, 
   D as the dimensionality of the training data points (always 2 for now), and
   K training epochs.

 The input file format is as follows:

    Line 1: A title for the display (text with no spaces)
    Line 2: Four numbers separated by spaces (or tabs): 
             P, D, H, and an arbitrary real number (in current version, 
	     this last number is ingnored).
    Lines 3 thru 3+P: 
	     The training data points and the class of each. One point per line.
             Each line must list all coordinates of each point then the class,
             all separated by spaces, like so:  Xcoord Ycoord class
    Lines 3+P+1 thru 3+P+K: 
             The weights and biases that determine the hyperplanes.  All the
             weights and biases for every node on the 1st hidden layer for a
             single epoch are listed on one line.  (i.e. one epoch per line.)
             The numbers will always be in the order: weightX1 weightY1 bias1.
             Thus, to display a net containing 4 hidden units use this format:
             wtX1 wtY1 bias1 wtX2 wtY2 bias2 wtX3 wtY3 bias3 wtX4 wtY4 bias4.

A sample from the 'demo3' data file included in this directory follows.
Notice that P=6 (number of training points), 
		  D=2 (must always be 2), and 
		  H=3 (number of hyperplanes).  And as always the number 
		      of data items
on a line for a single epoch is (D+1)*H, [in this case 30] items.
----------------------------------------------------------------------------


RANDOM_NET
6 2 3 1.0
.3 1 4
.3 .2 4
.3 .5 5
.5 .5 4
.8 .3 5
.8 1 5
   0.139    0.161   -0.071   -0.071    0.156   -0.184   -0.119    0.165    0.335 
  -0.033    0.032   -0.276   -0.152    0.067   -0.318   -0.249    0.047    0.151 
  -0.179   -0.095   -0.493   -0.223   -0.048   -0.515   -0.366   -0.093   -0.090 
  -0.281   -0.198   -0.669   -0.278   -0.154   -0.698   -0.461   -0.221   -0.308 
  -0.379   -0.280   -0.808   -0.341   -0.233   -0.833   -0.583   -0.332   -0.495 
   etc.

-------------------------------------------------------------------------------
DISCLAIMER:
  This software is distributed as shareware, and comes with no warantees
whatsoever for the software itself or systems that include it.  The authors
deny responsibility for errors, misstatements, or omissions that may or
may not lead to injuries or loss of property.  This code may not be sold
for profit, but may be distributed and copied free of charge as long as
the credits window, copyright statement in the program, and this notice
remain intact.

-------------------------------------------------------------------------------
References:

[Pratt, 1993]
@incollection{ pratt-93,
EDITOR          = "C.L. Giles and S. J. Hanson and J. D. Cowan",
BOOKTITLE       = "{Advances in Neural Information Processing Systems 5}",
AUTHOR          = "L. Y. Pratt",
TITLE           = "Discriminability-Based Transfer between Neural
		   Networks",
ADDRESS         = "San Mateo, CA",
PUBLISHER       = "Morgan Kaufmann Publishers",
YEAR            = 1993,
PAGES           = {204--211},
}

[Pratt, 1992]
AUTHOR          Lorien Y. Pratt
TITLE           "Non-literal transfer of information among inductive learners",
BOOKTITLE       Neural Networks: Theory and Applications
EDITOR          R.J.Mammone and Y. Y. Zeevi
YEAR            1992
PUBLISHER       Academic Press

[Ramachandran & Pratt, 1992]
EDITOR          John E. Moody and Steve J. Hanson and Richard P. Lippmann
TITLE           "Information Measure Based Skeletonisation"
BOOKTITLE       Advances in Neural Information Processing Systems 4
AUTHOR          Sowmya Ramachandran and Lorien Pratt
ADDRESS         San Mateo, CA
PUBLISHER       Morgan Kaufmann
YEAR            1992

[Mcclelland & Rumelhart, 1988]
TITLE           "Explorations in Parallel Distributed Processing: A
                 Handbook of Models, Programs, and Exercises"
AUTHOR          James L. McClelland and David E. Rumelhart
YEAR            1988
PUBLISHER       MIT Press
ADDRESS         Cambridge, MA



[Widrow & Winter, 1988]
TITLE          "Neural nets for Adaptive Filtering and Adaptive Pattern
                Recognition"
AUTHOR         Bernard Widrow and Rodney Winter
YEAR           1988
JOURNAL        IEEE Computer
MONTH          March

[Hanson & Burr, 1987]
AUTHOR          Stephen Jobs, Hanson and David J. Burr
TITLE           "Knowledge Representation in Connectionist Networks"
YEAR            1987
MONT            February
INSTITUTION     Bell Communications Research
