
			User Manual for Hyperplane Animator
				  Version 1.01
			  Lori Pratt and Steve Nicodemus
				October 11, 1993
			   lpratt@mines.colorado.edu


			      Table of Contents


1.  Introduction

2.  Walk-Through Example

3.  User Commands Reference

4.  Input File Format

5.  References


1.  Introduction
What is the Hyperplane Animator?
The Hyperplane Animator is a program that allows easy graphical display
of Back-Propagation training data and weights in a Back-Propagation
neural network.  It is not necessary to have an in-depth knowlegde of
neural nets to view the animator, and understand at least
superficially, the concepts behing neural net learning.

Back-Propagation neural networks consist of processing nodes
interconnected by adjustable, or ``weighted'' connections.  Neural
network learning consists of adjusting weights in response to a set of
training data.  The weights w1,w2,...wn on the connections into any one
node can be viewed as the coefficients in the equation of an
(n-1)-dimensional plane.  Each non-input node in the neural net is thus
associated with its own plane.  These hyperplanes are graphically
portrayed by the hyperplane animator.  On the same diagram it also
shows the training data. This approach has been used previously by many
researchers; the idea of plotting static hyperplane positions has been
presented by [Widrow & Winter, 1988], [Hanson & Burr, 1987], [Fahlman,
1990], and [Cohn et. al, 1990], among others.  The idea of animating
hyperplane movement was based on a program written by Paul Munro.

Why use it?
As learning progresses and the weights in a neural net alter,
hyperplane positions move.  At the end of the training they are in
positions that roughly divide training data into partitions, each of
which contains only one class of data.  Observations of hyperplane
movement can yield valuable insights into neural network learning.

General Description
The hyperplane animator requires a log from a network training session
as input.  In this file each weight and bias must have been recorded at
each epoch.  Thus the program displays not real-time training, but
instead shows animation of a previously trained network.  The user must
first use <FILE> from the menu to open a file previously created with
the correct format.  From there use any of the control buttons
described below to start or stop the animation, to control how
fast/slow the animation goes, and to adjust other display parameters
such as color and data point size.

About this version
This version was developed at Colorado School of Mines on an RS/6000,
using the Motif Widget toolkit.  Parts of the code, and parts of this
User Manual were taken from a previous version.


2.  Walk-Through Example

Type animator, and press <ENTER> to begin the program.
           The program opens a full screen window with a list of buttons 
           across the top.  

Click mouse on menu button marked <FILE>.
	   A File Browser window will appear.  Click mouse on the file
	   "demo3.txt".  Then click mouse on <OK>.  Window will
	   disappear. It may take up to 8 seconds to read a large data
	   file.  When the training data and set of hyperplane lines
	   appear on the screen, you may Start the animation.

Press <START>  to begin the animation.
	The training data points, (2 classes),  the coordinate axes,
	and ten hyperplanes will appear, (hyperplanes are represented
	simply as lines), and the hyperplanes will begin moving about
	the screen.  Press <PAUSE> if you want to "freeze" the
	animation.

	Note that each different class of training data is a different color. 

While the animation is continuing, or when it is paused, the user can
view or alter any of the controls by pressing the <PREFS> (preferences)
button.  Particularly, one may want to change INTERVAL to speed
animation, or the DELAY to slow it down.

This particular demo file contains 600 epochs and takes less than two
minutes to run on a reasonably free RS/6000.  Longer files may take up
to 4 or more minutes.

You can set up a continuous demo of this file by pressing the toggle
button "Demo" in the <PREFS> menu.

To exit the animator press the <QUIT> button.   To view a different
demo file, press <FILE>, choose file as described above, then press
<START>.


3.  User Commands Reference
-----------------------------------------------------------------------------
Basic Menu Options

FILE -  Produces a pop-up window which allows user to click open the
	desired file.   Opens the chosen animator file, which must be
	of the appropriate format, discussed below.  Reads all data
	from the file, which may require a delay of a few seconds.

QUIT - Quit animator.

START\REPEAT - Begin animation from 1st epoch.

PAUSE - Pause or "freeze" the animation.

RESUME - Resume animation from current epoch.

COLORS - User-define colors, and buttons to hide/show individual lines.
	Produces a pop-up window described on following pages.

PREFS - User-defined preferences, including line width, point size, etc. 
	Produces a pop-up window described on following pages.

CREDITS  - Produces pop-up window showing the design and programming
	history, and person to whom to direct comments/suggestion.

------------------------------------------------------------------------------
LEGEND & COLOR EDITOR Popup Window

This window is intended to function both as a legend for the hidden
units and training points, and a color editor to allow the user to
customize the colors.  Each hidden unit is labeled HU, and each
training point is labeled TP; similarly for the Axes and the Background
color.

To change the colors for any of the hidden units, training points, axes
or background, click the mouse within the label, then either move one
of the three slider panels to manually alter the color for the label
you last clicked on, or click on one of the numbered pixel squares at
the bottom of the window, to replace the color of the label with the
color of the pixel square.

Each hidden unit is initialized to the same pixel color, so clicking on
one label and altering that color will alter the colors of all hidden
units.  Remember though, that you may always choose other pixels for
any hidden units for which you desire a different color.

LOAD and SAVE buttons - Press SAVE to save the current color settings,
and LOAD to use (load from file) the last color settings which were
SAVEd.  To use this, there MUST be a file named "color.cfg" in the same
directory as the animator.  A file of that name is included with the
package.

Beside each hidden unit label is a toggle button for that label.  The
toggle button is to hide/show that hidden unit.  Click the button, and
the hidden unit will disappear.  When the button is "in", ie indented,
the hidden unit is "on" (shown).

Suggestions:  
The lower-numbered pixels are already in use by the application, and
any other applications, the window manager, etc.  If those pixels are
used, and then modified via the sliders, ALL other apps. which use them
will have altered colors.  To avoid unintentionally changing colors
already in use, pick the desired colors, then SAVE them and LOAD them.

It is not recommended to manually alter (eg with a text editor) the
values in "color.cfg" file.  If this file is missing, create a new one
with vi or any other plain text editor.  If LOAD does not work
properly, try SAVEing first and then loading.

------------------------------------------------------------------------------
PREFERENCES Popup Window

SHOW EPOCHS button - Produces a popup window in which the current epoch
      number is displayed.

MORE button - Produces a popup window with more, lesser-used
      preferences.  Described below.

Sliders
------- 
DELAY - Controls the amount of time between each Epoch
	(interval) of the animation.  Usually only small changes will be
	required to achieve smoothness of animation, (reduced flicker).

LINE WIDTH - Width of the hidden unit lines. 

POINT SIZE - Size of the training points.

Epoch Interval (next two scroll bars)
           BEGIN SKIP - At the interval shown in the label between the scroll 
		        bars, begin skipping epochs (intervals).
           INTERVAL SKIP - At the interval  (epoch) chosen with BEGIN SKIP, 
			skip the amount of intervals shown in the label.

MORE Popup Window - child of Preferences Window
-----------------------------------------------
Sliders
-------
X AXIS 
Y AXIS
  Use these sliders to place move your axes, lines and points within
  the drawing canvas.

COMPRESS - Use this slider to compress or expand your lines and points, 
   in relation to the screen coordinates.

DEMO toggle button - Demo mode will repeat the same animation, when it 
   reaches the end.
SHOW LABEL/POINTS toggle button - Toggle to show a rectangle (point) 
   for each training point, or the actual name of the class (label)
   specified k in the input file.

Suggestions:
1 - Preferences stay around as long as the application is open.  Sorry
    we don't have a reset button!  If you will be viewing various
    animations (different files), use COMPRESS sparingly.

2 - If you are primarily interested in only one animation file, you can
    use compress to expand the data to fill the entire canvas for easier
    viewing.

3 - The utility of Epoch Skip will become obvious after running a file
    with a large number of Epochs.  Such files will move very slowly after
    a certain number of intervals, since the network will be nearly
    trained.  Use Epoch Skip to speed up the animation at that point.

4 - As line width is increased, screen flicker may become a problem.
    Use small changes in delay to achieve smoothness; experiment with
    skipping every 2 or 3 epochs to regain speed lost by the increased
    delay.




4. Input File Format 

The hyperplane animator takes as input a data file representing the
results of neural network learning at each epoch.  Its file format is
described below; you'll need to modify whatever network training
program you use in order to accommodate this.  We use the software
distributed with:  [Mcclelland & Rumelhart, 1988] for our simulations,
and have developed shell scripts and conversion code for allowing this
program to generate the proper input format for the hyperplane
animator.

For an animation sequence involving:
   H as the number of hyperplanes (i.e. H nodes on the first hidden layer),
   P training data points, 
   D as the dimensionality of the training data points (always 2 for now), and
   K training epochs.

 The input file format is as follows:

    Line 1: A title for the display (text with no spaces) 
    Line 2: Four numbers separated by spaces (or tabs): 
             P, D, H, and an arbitrary real number.  (In the current version, 
	     this last number is ignored.  Be aware that if your data values 
   	     do not lie between -1 and 1  you may need to compress the 
	     data using the <PREFS> compress choice.)
    Lines 3 through 3+P: Training data points and their classes. One 
	     point per line.  Each line must list both coordinates of
	     each point, then the class, all separated by spaces, like
	     so:  Xcoord Ycoord class
    Lines 3+P+1 through 3+P+K: The weights and biases that determine 
	     the hyperplanes.  All the weights and biases for every
	     node on the 1st hidden layer for a single epoch are listed
	     on one line (i.e. one epoch per line).  The numbers will
	     always be in the order:  weightX1 weightY1 bias1.  Thus,
	     to display a net containing 4 hidden units use this
	     format:  wtX1 wtY1 bias1 wtX2 wtY2 bias2 wtX3 wtY3 bias3
	     wtX4 wtY4 bias4.

A sample from the 'demo3.txt' data file follows.  Note that P=6 (number
of training data), D=2 (should always be 2), and H=3 (number of
hyperplanes).  Note also that there are 3 floating point numbers for
each hyperplane, therefore 9 (3 times number of hyperplanes) floating
point number per line.

RANDOM_NET
6 2 3 1.0
.3 1 4
.3 .2 4
.3 .5 5
.5 .5 4
.8 .3 5
.8 1 5
   0.139    0.161   -0.071   -0.071    0.156   -0.184   -0.119    0.165    0.335 
  -0.033    0.032   -0.276   -0.152    0.067   -0.318   -0.249    0.047    0.151 
  -0.179   -0.095   -0.493   -0.223   -0.048   -0.515   -0.366   -0.093   -0.090 
  -0.281   -0.198   -0.669   -0.278   -0.154   -0.698   -0.461   -0.221   -0.308 
  -0.379   -0.280   -0.808   -0.341   -0.233   -0.833   -0.583   -0.332   -0.495 
   etc.



TALKING TO US:
  We're interested in hearing about your experience with this program.  Was
it useful, frustrating, fascinating, informative?  In exchange, we'll be
providing updated versions in the future.  Please send your questions as
well as requests for additions or deletions to/from our mailing list, to:
                      lpratt@mines.colorado.edu


DISCLAIMER:
  This software is distributed as shareware, and comes with no warantees 
whatsoever for the software itself or systems that include it.  The authors 
deny responsibility for errors, misstatements, or omissions that may or 
may not lead to injuries or loss of property.  This code may not be sold 
for profit, but may be distributed and copied free of charge as long as 
the credits window, copyright statement in the ha.c program, and this notice 
remain intact.

