


Advanced Network Simulator In Lisp


(ANSIL) 


User's Manual


Version 0.95 





Peter J. Angeline and Gregory M. Saunders





The Ohio State University


Laboratory for Artificial Intelligence Research (LAIR)


Columbus, Ohio, 43210





			email: ansil@cis.ohio-state.edu





			 Copyright ? 1991





				 Draft


1.0  Introduction and Organization


1.1  License and Disclaimer


License: This code may be distributed free of charge. This code may not be incorporated into any 
production software of any type or any code which is sold without the express written consent of 
the authors. Modifications and inclusions for other types of software are permitted, provided that 
the headers remain unchanged and included in all subsequent versions of the code.


Disclaimer: The authors of this code make no claims as to its validity, and are in no way responsi-
ble for any errors contained within.


1.2   What is Needed to Run ANSIL 


There are really two version of this code included in this release, one for generic common lisp and 
one for Franz Allegro Common Lisp. At a minimum, you need access to some version of common 
lisp. It is preferable that Franz Allegro Common Lisp be used since this version of the code is at 
least 10 times faster than the generic common lisp version. However, it is not necessary to have 
Franz's common lisp to use the ANSIL system.


1.3  Files Included in This Release


o	    	Readme - general help to get started.


o	    	compile.lisp - file which will compile system automatically from lisp.


o	    	go.lisp - file to load ANSIL.


o	    	linear.lisp - linear algebra routines in lisp


o	    	clinear.lisp - an optimized linear algebra package calling C routines 		 from lisp


o	    	linear-lib.c - C routines for clinear.lisp


o	    	matrixbp.lisp - matrix implementation of backprop using linear algebra 			routines. 


o	    	raam1.lisp - matrix implementation of RAAMs using backprop in 		 matrixbp.lisp. This 
file contains code for setting up and 		 training raams.


o	    raam2.lisp - raam maintenance - viewing, saving, and analyzing raams.


o	    	ansil.manual.txt - an ascii version of this document.


o	    	ansil.manual.ps - a postscript version of this document.


o	    	syntax.lisp - an example of a RAAM definition which runs in this code. 		 This example 
was the syntax example from Jordan 		 Pollack's paper on RAAMs. 


o	    	sentences.lisp - an example of a RAAM definition. This example is the 		 sentences 
example from Jordan Pollack's paper on 			 RAAMs.


o	    letseq.lisp - an example of a large stack implemented in a RAAM. This raam encodes 
stacks of letters which form words using the letters "b" "r" "a" "i" "n". This is also 
from Jordan's AI Journal paper.


o	    shiftreg434.lisp - another stack example from the RAAM paper. 


o	    xor.lisp - an example of a definition of a network. This is the 2-2-1 exclusive-or exam-
ple from the PDP books.


2.0  Compiling and Loading ANSIL


2.1  Obtaining ANSIL


 ANSIL is available by anonymous ftp only. We currently have no method of distributing ANSIL 
in any other fashion. If you have trouble following the procedure outlined below, send mail to 
ansil@cis.ohio-state.edu for assistance. 


To obtain a copy of ANSIL from any unix system, do the following:


unix> ftp nervous.cis.ohio-state.edu


ftp> binary


ftp> cd pub/ansil


ftp> get ansil.tar.Z





The retrieved file is both compressed and tared to minimize to size of the ANSIL code and docu-
mentation to allow quicker transfer time during retrieval by FTP. First the retrieved file must be 
uncompressed with the unix command uncompress in the following manner:


uncompress ansil.tar.Z


This will uncompress the file and leave in its place the file ansil.tar. This is a unix tar file which 
holds not only the files associated with the ANSIL system but the directory structure which orga-
nizes those files as well. To restore the directory structure and all the associated files for ANSIL, 
you must first create the root directory where ANSIL will exist, copy the ansil.tar file to that direc-
tory and untar the file. The directory structure and associated files are reconstructed with the com-
mand:


tar xvf ansil.tar


Make sure that the file has been uncompressed before running it through the tar program. For 
instance, if I wanted the ANSIL system to reside in the directory /usr/ansil and I just ftp'd it into 
my personal directory in /pete, I would do the following:


unix> cd /usr


unix> mkdir ansil


unix> cd ansil


unix> mv /pete/ansil.tar.Z .


unix> uncompress ansil.tar.Z


unix> tar xvf ansil.tar


After the program has completed, the ansil directory will have three subdirectories: source, docs 
and examples. The source directory holds all of the lisp code and must be compiled for your par-
ticular lisp. The docs directory holds the ANSIL manual (this document) in both a postscript ver-
sion and a text version. The examples directory holds several examples showing how to use the 
various features of ANSIL.


2.2  Compilation 


The file COMPILE.LISP has been included to automatically compile the files for the correct sys-
tem. You must first alter the file to give the correct pathname so the files can be located to be com-
piled. The line


(setf *ansil-path* "/n/nervous/1/ansil/source/") 


must be altered so the correct pathname is between the double quotes. Be sure the path is the cor-
rect format for the operating system you're using. For UNIX systems, this means making sure the 
path specifies a directory and ends with a "/". After this is done, the correct fasl-extension must be 
provided a few lines down in the line which reads 


(setf *fasl-extension* ".fasl")


which is the extension for Franz Allegro Common Lisp. After both of these have been done, sim-
ply load your version of common lisp and load the compile.lisp file. When the "Compilation 
Complete" message has been printed to the screen the code is ready to be compiled and loaded.


The code in the ANSIL system uses a collection of macros called the LOOP package. If your sys-
tem does not have the loop package installed, the code will not work. Contact your system admin-
istrator and/or lisp vendor to acquire this package. If all else fails, send mail to the account 
described in the last section of this manual with a request for the loops package. 


2.3  Loading the ANSIL System


Once you have made the changes to the COMPILE.LISP file and compiled the ANSIL system, 
you should make the same changes to the GO.LISP file as well. The GO.LISP file will automati-
cally load the correct system for the common lisp you are using. Thus, if you are using Franz 
Allegro Common Lisp, the fast implementation will be loaded with access to the linear algebra 
routines written in C, which is described in the next section. To load the system after it has been 
compiled, simply start a common lisp process and load the go.lisp file. 


3.0  The Linear Algebra (LINEAR) Package 


This package is a support package for the rest of the code. It contains some typical linear algebra 
routines and some routines which are specific to backprop but are matrix oriented. There are two 
versions of this code, one in the file LINEAR.LISP and one in the file CLINEAR.LISP. Both ver-
sions implement the same routines but do so in different manners as described below. Both ver-
sions of this code load their routines into the package LINEAR. Be sure not to attempt to load the 
CLINEAR.LISP code unless you have Franz Allegro Common Lisp version 3.1 or higher. See the 
instructions in the section "Loading the ANSIL System" for further information.


3.1  Generic Common Lisp Version 


The file LINEAR.LISP contains the generic common lisp for all the linear algebra and matrix 
manipulation routines that are required by our version of backprop and the ANSIL packages. This 
package should run under any implementation of common lisp with little or no modification. This 
code however, is rather slow in comparison to the same type of code written in C due to the usual 
overhead of LISP. 


3.2  FCL/C Hybrid Version


 In order to improve the processing speed of the package, and since all of the real time consuming 
routines reside in this package, we have recoded the LINEAR package as a collection of C rou-
tines which are stored in the file LINEAR-LIB.C. Franz Allegro Common Lisp has a facility 
called The Foreign Function Interface which allows the calling of C routines from within LISP. 
The routines contained in CLINEAR.LISP are just LISP definitions which agree with those in 
LINEAR.LISP but instead of performing the operations in LISP, these routines call the equivalent 
C functions. It is not uncommon to experience a 10 fold speed-up using CLINEAR.LISP rather 
than LINEAR.LISP. Note that this can only be done with Franz Allegro Common Lisp.


4.0   The Matrix Backprop (MPB) Package 


The current version of the MBP package assumes that the network will be feed-forward only and 
that the connections will be between consecutive layers of the network only. While this poses a 
severe limitation on the types of networks which can be constructed, it minimizes the computation 
needed for RAAMs. It is expected that in future versions this will be amended.


 Before the MBP package is ready to be used, be sure that the LINEAR package has been loaded. 
Also, insure that the MBP package is being "used" meaning that the routines which are apart of 
the MBP package are available for use in the current package. This is accomplished with the fol-
lowing call: 


(use-package :mbp)


which makes the exported routines in the MBP package available for use in the package you are 
currently in. For further information on packages and the use-package function, see Common Lisp 
The Language by Guy Steele. Each of the example files provided uses the use-package command, 
you can look there for an example. 


4.1  Creating NETWORKs


 There is a single routine which creates a network. CREATE-NETWORK has the following syn-
tax:


	(CREATE-NETWORK <levels>+)


where <levels>+ is a sequence of 2 or more integers which describe the number of nodes at that 
layer of the network. For instance, the call:


	(setf net (CREATE-NETWORK 2 2 1))


sets up a 2 input, 1 output network with 2 hidden units. The return value for the function is the 
network structure itself and must be stored in a variable for future usage in other ANSIL func-
tions. The setf above places that return value into the lisp variable NET. 


After a network has been defined, we need to define the input-output pairs that it is to associate. 
This is done with the ADD-PATTERN routine, which has the following syntax:


	(ADD-PATTERN <network> <in-list> <out-list>)


where <network> is the structure returned by the CREATE-NETWORK function, <in-list> is 
either a list or a vector of numbers and <out-list> is either a list or vector of numbers. Each call to 
ADD-PATTERN signifies an input-output pair to be learned by the network. The <in-list> must 
agree in length with the input length of the network while the <out-list> must agree in length with 
the output length of the network. For example:


	(setf xor (create-network 2 2 1))


	(add-pattern xor `(1 1) `(0))


(add-pattern xor `(1 0) `(1))


	(add-pattern xor `(0 1) `(1)) 	


	(add-pattern xor `(0 0) `(0))


defines a network and input-output patterns for a 2-2-1 exclusive-OR network.


4.2  Altering NETWORKs with SET-NETWORK 


An MBP network consists of several parts which are modifiable by the user. They include:


o	    rate - [Default 0.7] The learning rate for the network. 


o	    	momentum - [Default 0.5] The momentum previous updates of the weights have. 


o	    	epsilon - [Default 0.1] How close each output must be to the target value before it 		 is 
considered to be learned. This is also called tolerance.


o	    	bias - [Default 1.0] The constant activation of the bias node.


o	    	adjust? - [Default nil] When this flag is T the rate is automatically divided by the 	 num-
ber of patterns being learned. Setting rate to 1.0 and 		 making this flag T turns out to be a 
good guess for a rate 		 but may be slower than the optimal rate. It will insure a 		 non-
oscillatory trajectory for the weights.


o	    	sp-add - [Default 0.0] Scott Fahlman described a trick which adds a constant to the 	 
derivative of the sigmoid to speed up learning. This variable 		 is the value to be added. 
The default is 0.0 to get pure 		 backprop. A good value is usually 0.1. 


o	    	always-bp? [Default T]- Sometimes, it is beneficial to not backprop nodes which 		 are 
within tolerance. This flag states whether to 		 backprop an error from a node within tol-
erance.


All of these parameters can be changed for a network via the function SET-NETWORK which 
has the following syntax: 	


(SET-NETWORK <network> :<key> <key-value>)>


where <network> is the variable holding the returned structure from CREATE-NETWORK, 
<key> is one of the words described above and <key-value> is the new value for that aspect of 
the network. Note that each keyword must be preceeded by a colon (this is common lisp syntax). 
Any number of :<key> <key-value> pairs can be given. For example:


	(setf my-net (create-network 2 3 4 5)) 


	(SET-NETWORK my-net :rate 1.0 :momentum 1.0 :adjust? t)


is valid and alters the rate, momentum and adjust? values for the network structure stored in the 
variable my-net. If the call to SET-NETWORK is made interactively, the current values of the 
network will be displayed after the requested alterations have been made. A call to SET-NET-
WORK with only a <network> provided as a parameter is valid syntax and will do nothing to the 
network structure. Such a call when done interactively will print out the parameters of the net-
work structure.


4.3  Training NETWORKs with NETWORK-TRAIN


 Once a network has been setup as desired, we can use the function NETWORK-TRAIN to per-
form backprop over the defined network. The syntax for NETWORK-TRAIN is as follows:


		(NETWORK-TRAIN <network>)


where <network> is the network structure. During the training of the network, the following 
information is printed to the screen:


	**.. Epoch 123: Err=3 Tot=1.003 Max=0.42011


This information is printed after each epoch to describe what is going on with the network. The 
leading characters show which patterns have converged using a "*" for each pattern out of toler-
ance and "." for each pattern within tolerance. The numbers following give the current epoch 
number, the total number of units outside of tolerance for all patterns, the total square error of the 
last comparison and the maximum absolute error over all nodes and patterns. 


 The printing of this information can at times significantly slow the operation of the code. The 
keyword :display has been supplied in the NETWORK-TRAIN function for specifying how often 
this information is to be printed. It is used as follows:


	(NETWORK-TRAIN my-net :display 10)


which will only display the epoch information every 10 epochs. The default for this keyword is 1 
which prints information after each epoch. A value of 0 for this keyword will only print out infor-
mation at the conclusion of the training run. 


 Occasionally a network will never converge or converge very slowly for various reasons. To pre-
vent the training code from running indefinitely, a second keyword has been added to supply a 
max number of epochs to run before the program gives up. This keyword is :num and has a 
default of 10,000 epochs. It can be altered in the initial call to the NETWORK-TRAIN routine as 
follows:


		(network-train my-net :display 10 :num 50000)


In this call, 50,000 epochs of backprop will be performed before the routine quits trying. If all of 
the nodes for all of the patterns are within tolerance prior to reaching this limit, the routine will 
finish as well.


4.4  Using a Network


Once a network has been converged, you can pass an arbitrary vector through the network and see 
what the network returns with the function USE-NETWORK. This function has the following 
syntax:


(use-network <network> <input>)


where <network> is a network structure and <input> is either a list or a vector of values to be 
assigned to the input of the network. The return value is of the same type as provided and is the 
output of the network when the input given is passed through.


4.5  Displaying NETWORKs with SHOW-NETWORK 


SHOW-NETWORK is mostly used just to see the weights and is not a very useful function at this 
point. The syntax is:


		(SHOW-NETWORK <network>)


where <network> is the network structure returned by create-network.


4.6  Saving and Restoring NETWORKs


 To save a network to a disk file, the following function is used:


	(WRITE-NETWORK <network> <name>)


where <network> is the network to be saved and <name> is the file name without an extension. 
The filename may include a pathname if the default directory is not the desired place for the file. 


 To read in a network saved with the above function, simply use 


		(READ-NETWORK <name>)


where <name> is the same as above. The return value of this function is the network structure 
and must be saved in a variable as with the create-network routine. 


5.0  The Matrix RAAM (MRAAM) Package 


The package MRAAM uses both the LINEAR package and the MBP package to implement a ver-
sion of RAAMs.


5.1  Creating RAAMs with CREATE-RAAM 


The function to create a RAAM is rather full of features. The function CREATE-RAAM at it's 
minimum is simply just (CREATE-RAAM) which will return a RAAM structure with all of the 
features filled in with defaults. Each of the following keys can be used to change the value of that 
feature from the default into a desired value:


o	    	trees - [Default `()] This is the feature for describing what trees 		are to be learned by the 
RAAM.


o	    	valence - This is the branching factor of the trees. 


o	    	net - [Default nil] If this is nil then a new network is created and 		placed in the RAAM 
structure, otherwise a network structure 		must be supplied. We recommend that this fea-
ture not be used.


o	    	terminal-codes - [Default nil] This parameter is one way you 		can set values for the ter-
minals. Syntax is as in this 		example: 		 (create-raam :terminal-codes `((a (0 0)) (b (0 1)) 
...) 		In other words, a list of <terminal, representation> pairs. 		An alternate method for 
setting the value of a terminal is the 		function SET-TERMINAL which is described 
below.


o	    	rep-width - The number of nodes to dedicate to a single terminal or 	non-terminal repre-
sentation.


o	    	term-tol - [Default 0.2] The tolerance that will be used to determine 		if a terminal is 
decoded correctly.


o	    	nonterm-tol - [Default 0.05] The tolerance that will be used to 		determine if a non-ter-
minal is decoded correctly.


o	    	name - [Default "GENERIC-RAAM"] Name for the RAAM. Must be a string.


o	    	random-weights - [Default t] Flag whether to randomize the weights of 		the network in 
the RAAM structure or not. Should always be T.


o	    	random-terminals - [Default nil] Will assign random values for 		terminals if set to T. 


o	    	rate - [Default 1.0] The learning rate for the network which 		implements the RAAM.


o	    	momentum - [Default 0.5] The momentum for the RAAM network.


o	    repeat? - [Default t] When this variable is t the statistics of the training set are pre-
served in the learning of the raam. Thus if a tree or subtree appears 10 times in the 
training set then the training will be done as that tree or subtree is backpropped 10 
times. If the variable is nil then each tree and subtree is weighted the same in the train-
ing. There is no speed difference between these two choices but the resulting generali-
zations are usually quite different.


o	    	sp-add - [Default 0.1] A constant amount to add to the derivative of 		the sigmoid. When 
this is non-zero and 	small backprop is a bit faster.


o	    	adjust? - [Default nil] When this is T the rate is divided by the 		number of patterns 
being learned. It has been found that by 		setting the rate to 1.0 and allowing this to be T 
is a good 		guess for a rate for a RAAM. 


o	    	always-bp? - [Default t] If T then backprop is always done. If nil 		then if a node is 
within tolerance of the target it will not 		be backproped.


o	    	nil-value - [Default 0.5] The trees given to create-raam will 		automatically be converted 
to have the valence specified in the variable "valence", described above. To do this, a 
"nil" symbol is needed, whose representation will be a vector 	 of length rep-width with 
each element set to this value.


Most of these options you will not need and are provided to integrate this code with larger lisp 
programs which use RAAMs as a component. Each of these are used as keys in the CREATE-
RAAM call. For instance:


(create-raam	:trees `((a b) (a c) ((a b) (c d))) 		 	 


 		:rep-width 2 			 :valence 2 			


	:terminal-codes 


	 `((a (0 0)) (b (0 1)) (c (1 0)) (d (1 1))) 			 


	:momentum 0.7 			 :adjust? t)


will create a RAAM structure which will learn trees with a branching factor of 2 and a representa-
tion width of 2, thus a 4-2-4 RAAM. This RAAM will learn the trees (A B), (A C) and ((A B) (C 
D)) where A, B, C and D are represented as described above. Also, the momentum has been 
changed from the default to 0.7. See the code for the examples included with ANSIL for further 
examples.


 An alternate method for defining the values for terminals is the SET-TERMINAL function. The 
syntax is:


	(SET-TERMINAL <raam> <term> <term-val>)


where <raam> is the structure returned by the CREATE-RAAM function, <term> is the name of 
the terminal as it appears in the trees and <term-val> is a list of length equal to the rep-width of 
the RAAM and is composed of values between 0 and 1 inclusive. For instance, we could redo the 
above RAAM as follows:


	(setf my-raam 


 (create-raam :trees `((a b) (a c) ((a b) (c d)))


 		 :rep-width 2 :valence 2 :momentum 0.7 :adjust? t))


	(set-terminal my-raam `a `(0 0)) 	


(set-terminal my-raam `b `(0 1)) 	


(set-terminal my-raam `c `(1 0)) 	


(set-terminal my-raam `d `(1 1))


which has more syntax but my be more readable to some users. The examples have been coded in 
this format. 


5.2  	Altering RAAMs with SET-RAAM


 Some of the features in the CREATE-RAAM call can be modified after the RAAM has been cre-
ated. These features are: rate, momentum, term-tol, nonterm-tol, name, always-bp?, adjust?, 
repeat? and sp-add. These alterations can be done with the function SET-RAAM with the follow-
ing syntax:


		(set-raam <raam> :<key> <key-value>)


where <raam> is the structure returned by the CREATE-RAAM routine, <key> is one of the fea-
tures mentioned and <key-value> is the new value for the key. There can be as many <key> 
<key-value> pairs as necessary. So, we could alter our RAAM defined above in the following 
way:


		(set-raam my-raam :rate 0.5 :adjust? nil)


which will change the rate to be 0.5 and stop doing the adjust trick outlined above. If no <key> 
<key-value> pairs are given, SET-RAAM will print out the current settings for the features for the 
RAAM. 	


5.3  Training RAAMs with RAAM-TRAIN 


For the most part, the training of a RAAM is basically no different than a typical usage of back-
prop. However, there are two very important issues which apply to RAAMs and not to networks 
in general. These issues are how to detect if the information decoded is a terminal or non-terminal 
and how to determine that the RAAM has adequately represented the desired trees. The first issue 
is solved by including a Terminal Test into the RAAM. The second issue is solved by defining a 
Termination Test for the RAAM training. Before describing how to train a RAAM, we must 
describe the Terminal Test and the various Termination Tests that have been incorporated into the 
code.


5.3.1  The Terminal Test


When reconstructing a tree from its compressed representation, we need to determine whether or 
not an uncompressed representation is a terminal node or a non-terminal node for the RAAM. If it 
is a terminal node, then that branch of the tree has been fully decoded. If it is a non-terminal code, 
then that code must be passed through the decoder again until all terminals are found. We would 
like to avoid the inelegant method of testing each child decoded against what we know should be 
there and rely fully on the RAAM network for this information. To achieve this, an output unit is 
added to the RAAM for each branch of the tree. The purpose of the decoding section of the 
RAAM is now to both decode the representation correctly and make the corresponding bit in the 
output layer greater than 0.5 if the decoded child is a terminal and less than 0.5 if the decoded 
child is a non-terminal. Thus, when reconstructing the tree, only the corresponding bit for the 
child need be inspected to determine if the child is a terminal or non-terminal. The terminal test 
can be disabled using the key :terminal-test in the raam-train function.


5.3.2  The Termination Tests


 A simple minded termination test, and an easy one, is to simply train the RAAM until all the ter-
minals are within a specific tolerance and all of the non-terminals are within a specific tolerance. 
Thus, just as in normal backprop, we place an input on the input nodes of the RAAM, it encodes 
into the hidden representation and decodes back to the original inputs on the output layer. Usually 
this is adequate to learn the desired trees and serves as the default termination test in our code. 
Note that it is very important in this termination test to decode the non-terminals as exactly as 
possible to avoid cumulative error buildup during reconstruction.


 However, it may be the case that the non-terminals will not reconstruct properly even though they 
are within tolerance when trained. The only method we know of to ensure the trees are recon-
structed correctly is to check that they all can be reconstructed from their hidden unit representa-
tions. So, a second termination test which halts the training process only after the trees can be 
reconstructed has been supplied. This termination test is selected when the call to train the RAAM 
is made by a positive, non-zero value for the key :reconstruct. Usually, this termination test results 
in fewer epochs being needed for training however, the reconstruction of the trees takes quite 
some time.


5.3.3  Training a RAAM with RAAM-TRAIN


 The function RAAM-TRAIN performs backprop with the modifications described above on a 
RAAM structure. The syntax for the RAAM-TRAIN function is simply:


		(RAAM-TRAIN <raam>)


There are four keys which go along with this call:


o	    	display - [Default 1] If the value of this feature is n then the 		information about the train-
ing of the RAAM is displayed after 		every n epochs. It is recommended that n be around 
10 or 		greater for long runs since reducing the printing of information 		can greatly reduce 
the overall execution time


o	    	num - [Default 10000] This is the number of epochs to run before the training 		routine 
gives up.


o	    	reconstruct - [Default 0] When this is 0 then the reconstruction 		termination test is not 
used to determine when the RAAM has 		finished training. Instead the simple tolerance 
test is used. 		If this feature is non-zero, then the reconstruction terminal 		test is used and 
the value determines how often the trees are 		reconstructed. It is recommended that the 
value of this 		feature be no less than 10 since the time needed to 		reconstruct the trees is 
usually quite large. When this key is non-zero, the terminal-test key must also be t.


o	    	terminal-test - [Default t] When this is t the terminal bits are learned as well as the 
given raam. It should be noted that a RAAM can be within tolerances but not pass the 
terminal test for all subtrees. This will prevent the training procedure from halting. 
When this variable is nil, the terminal bits are ignored when computing errors and are 
not a criterion for convergence. Whenever the reconstruct key is non-zero, this parame-
ter must be t.


So, we could run our RAAM defined above with the following command:


	(RAAM-TRAIN my-raam :num 5000 :display 10 :reconstruct 10)


which will run for a max of 5000 epochs, display information about the training of the RAAM 
every 10 epochs and test for termination by reconstructing all the trees every 10 epochs. The com-
mand:


	(RAAM-TRAIN my-raam :num 5000 :display 10 :terminal-test 
nil)


will run as above without both tree reconstruction and the terminal test and will complete the 
training when the terminals and non-terminals are reconstructed within the tolerances specified 
for the raam.


5.4  Displaying RAAMs with SHOW-RAAM


 Show-raam is the routine for viewing the raam. It can be called with the syntax: 


	 (SHOW-RAAM <raam>)


For instance, suppose we have the same simple raam that we used above:


	(setf my-raam 


 (create-raam :trees `((a b) (a c) ((a b) (c d))) 		 


 		 :rep-width 2 :valence 2 :momentum 0.7 :adjust? t))


	(set-terminal my-raam `a `(0 0)) 


	(set-terminal my-raam `b `(0 1)) 	


(set-terminal my-raam `c `(1 0)) 


	(set-terminal my-raam `d `(1 1))


 After training to tolerance, we call show-raam:


	(show-raam my-raam)


	=====================================================


 	GENERIC-RAAM, created Wed Jan 16 10:54:18 1991 	
=====================================================





Parameters 	 


 	valence = 2 	 


	rep-width = 2 	 


	raam-width = 4 





Training information 


	*Rate 	= 1.000000 	 


	*Momentum 	= 0.700000 	 


	*Terminal tolerance 	= 0.200000 	


	*Nonterminal tolerance 		= 0.050000 	 


	Total error 	= 0.000000 	 


	Max error 	= 0.000000 	 


	Total epochs 	= 0


	


Symbol table 	 


	NIL	= #( 0.500 0.500) 	 


	A 	= #( 0.000 0.000) 	 


	B 	= #( 0.000 1.000) 	 


	(A B) 	= #( 0.000 0.974) 	 


	C 	= #( 1.000 0.000) 	 


	(A C) 	= #( 0.989 0.000) 	 


	D 	= #( 1.000 1.000) 	 


	(C D) 	= #( 0.995 0.995) 	 


	((A B) (C D)) 	= #( 0.000 0.000)





Training set 	 


	((A B) . 3) = (#(-0.051 -0.028) (#(-0.044 0.000))


	((A C) . 5) = (#(-0.050 -0.027) (#(0.000 -0.044)) 	


	((C D) . 7) = (#( 0.065 -0.000) (#(0.023 0.024)) 





	(((A B) (C D)) . 8) 


		= (#(-0.000 0.050) (#(0.038 0.036) )


Most of the above information is self-explanatory. The only difficulty might be in the training set, 
directly above. Take the first line, for instance. ((a b) . 3) means that the tree (a b) is part of the 
training set, which is stored as the 3rd item in the symbol table. The vectors to the right indicate 
how well this tree is decoded. When we try to decode (a b), we want to get back the representation 
for a and the representation for b. The vector above indicates the difference between the target 
value for each subtree (from the symbol table), and the actual value (from decoding the current 
tree). For instance, if we decode (a b), the goal for the left subtree is #(0.000 0.000), the represen-
tation for a in the symbol table. When we actually decode (a b), the left subtree is #( 0.051 0.028), 
giving a difference of #( -0.051 -0.028). Similarly, the difference between the target value of b and 
the actual value of b gives a difference of #( -0.044 0.000).


There are lots of options to show-raam, all specified by keyword parameters


o	    max-symbol-length <n> - only prints the first n characters of each symbol


o	    digits <n> - only prints n digits to the right of the decimal point


o	    width <n> - limits width of print field to n 


o	    verbose T/nil	 - prints extra information on the raam, including weights


If the weights are printed by setting verbose to T, the following additional keywords can be used:


o	    max <n> - maximum weight which will be printed


o	    min <n> - minimum weight which will be printed


o	    threshold <n> - entries will be rounded to min/max if within threshold


Any entry above max will be printed as a "+". Similarly, any entry below min will be printed as a 
"-". 


5.5  Saving and Restoring RAAMs


 In the event that you'd like to save your trained RAAM for future use, there is a routine to write 
it out to a file. WRITE-RAAM works as follows:


		(write-raam <raam> <filename>)


will write a lisp readable description of the RAAM in <raam> to the file specified in <filename>. 
<filename> should be a string which includes the path if necessary. The format which gets written 
to the file is pretty printed so it can be read by humans more easily. For larger RAAMS, this can 
take quite some time.


 A RAAM which is written to a file by the WRITE-RAAM routine can be read back in via the 
READ-RAAM function as follows:


		(read-raam <filename>)


where <filename> is a string which describes which file contains the RAAM to be loaded. The 
return value of this function is the RAAM structure and thus must be saved in a function.


5.6  Encoding & Decoding After Convergence


Once a RAAM has been trained, we would like to do some things with it. We have provided some 
simple encoding and decoding routines which can be used for various purposes.


The ENCODE function takes a symbolic version of a tree and returns the vector of values 
encoded by the RAAM for that tree. For instance, assume I have trained a RAAM and have it 
stored in the variable my-raam. Now I wish to observe what the compressed representation for the 
tree `((a b) c) which was not in the original training set. This is done as follows:


	(encode my-raam `((a b) c))


The return value of the call is a vector which is the compressed representation of this tree in the 
RAAM provided. This value can be passed to RECONSTRUCT-TREE to determine if the 
RAAM compressed the tree correctly. We use this method to determine how well a RAAM gener-
alized over a particular training set.


 There are two functions available for reconstructing trees after the RAAM has finished training. 
Each of these functions does something slightly different than the other. For instance, RECON-
STRUCT-TREE takes an arbitrary vector of a compressed representation and uses the RAAM to 
reconstruct the symbolic version of the tree. It's syntax is:


(reconstruct-tree <raam> <vector>)


where <raam> is the trained RAAM and <vector> is a vector of values which is the same length 
as the rep-width of the RAAM. This function uses the Terminal Test to determine when it has 
uncompressed a particular branch entirely. Which terminal the branch has uncompressed into is 
determined by a NEAREST NEIGHBOR search. **NOTE** :: When using the Lisp/C hybrid 
version of ANSIL, this vector must be a vector of type DOUBLE-FLOAT to interface correctly 
with the C code. 


The function RECONSTRUCT-TREES takes a RAAM as its single parameter and tries to recon-
struct all the trees in the training set. Note that this function is very different from the :reconstruct 
option available with RAAM-TRAIN. The :reconstruct option checks to see if the trees in the 
training set can be reconstructed TO WITHIN THE TERMINAL TOLERANCE provided in the 
RAAM definition. RECONSTRUCT-TREES checks to see if the trees in the training set can be 
reconstructed by using a NEAREST NEIGHBOR search through the available terminals of the 
RAAM. Hence it's possible that the RECONSTRUCT-TREES routine can reconstruct all the trees 
even though the RAAM fails one or both of the Termination Tests. 


There are 2 keys which can be used with either of the reconstruct routines:


o	    	 max-depth - [Default 30] provided to limit the depth the routines will search for termi-
nals when uncompressing a tree.


o	    symbolp - [Default t] When t the return value is a tree of defined terminal. When nil the 
return value is a tree of the actual vectors which appeared on the outputs.


There is an extra key for RECONSTRUCT-TREES called pprint which will pretty print the output 
of each reconstruction so it is a little easier to read.


6.0  Running the Examples 


All examples are from Jordan Pollack's paper in Artificial Intelligence entitled "Recursive Auto-
Associative Memory". Each can be executed by first loading the ANSIL system and then loading 
the desired file. These files are designed to first define the RAAM then train it. Comments are pro-
vided in each file to give the user a feel for how many epochs it will take to converge. Also, by 
looking at the files a sense of what parameters we have found to be most useful and their values 
can be determined. 


7.0  Other Functions


There are many functions implemented within the code that have importance when building sys-
tems which use the ANSIL software. In this section we will list many of the functions including 
which package and file they reside in. These functions are to be considered volatile, in that their 
functionality, implementation and specification may be drastically altered from version to version 
according to our own random whims. 





Function	Parameters	Description


mbp:randomize-network	net 	Randomizes the weights in the given network. 


mbp:randomize-vector	vect	Randomize the given vector.


mbp:set-nodes	net pat level	Sets the activation of the level to be the given pattern. 
Level is optional and defaults to 0 the input level. 


mbp:get-nodes	net level	Returns a vector with the activations of the nodes of 
the asked for level. Level is optional and defaults to 
the output layer.


mbp:revert-network	net	Restores the saved initial weights of a network.


mbp:show-patterns	net	Show the current associations of the network.


mbp:use-network	net pat	Run a pattern through the network and return the out-
put.


mraam:randomize-raam-network	raam	Randomize the initial conditions of the raam.


mraam:revert-raam-network	raam	Revert a raam to the stored initial conditions.


mraam:verify	raam	Test the trees in the training set and return T if it can 
be reconstructed from it's compressed representation. 
Stops as soon as one is found which can't be recon-
structed.


mraam:uncompress	raam code	Use the current decoder of the raam to get the children 
of the given code.


mraam:encode	raam tree	Takes the symbolic version of a tree and returns the 
compressed representation as a vector.


8.0  Undocumented Features and Functions


 There are many features which have been incorporated in the ANSIL software which we have yet 
to document. Users are encouraged to read the code and the surrounding comments in order to 
locate these features. The code is rather verbosely documented for just this reason. We feel that if 
you've mastered everything in this manual you should be able to move to the comments in the 
code and understand what is going on for the most part. A good indication of a useful or important 
function is it's presence in the export command in each of the files. By surveying the functions 
which appear in each of these, you should be able to garner a feel for what we have found useful.


9.0  User Assistance and Support


When you receive this software, please send your name and email address to ansil@cis.ohio-
state.edu as registration. We will keep you informed about upgrades and additions to the ANSIL 
system. This will also be a vehicle for us to begin our RAAM User's Group (RUG) where we can 
trade suggestions, hacks and ideas. More on this in future versions.


 If you have a bug report, a comment about the code or an addition you would like to see incorpo-
rated with the next release, you can contact the ANSIL User Assistance Group at the email 
address ansil@cis.ohio-state.edu. Also, if you expand on this code or improve it, please let us 
have a copy so that we can incorporate it into the distributed version. Any and all comments, 
hacks or suggestions are gratefully accepted.



