From xerion-users-owner Tue Jan 19 18:37:38 1993 Received: from usc.edu ([128.125.253.136]) by relay.cs.toronto.edu with SMTP id <150239>; Tue, 19 Jan 1993 18:37:05 -0500 Received: from gizmo.usc.edu by usc.edu (4.1/SMI-3.0DEV3-USC+3.0) id AA02074; Tue, 19 Jan 93 15:36:56 PST Received: by gizmo.usc.edu (4.1/SMI-4.1+ucs-3.5) id AA06662; Tue, 19 Jan 93 15:36:55 PST Date: Tue, 19 Jan 1993 18:36:55 -0500 From: mharm@gizmo.usc.edu (Mike Harm) Message-Id: <9301192336.AA06662@gizmo.usc.edu> To: xerion-users@ai.toronto.edu Subject: wobbly function? Hi. I just modified rbp to allow randomness in picking examples, so that some examples are trained against more frequently than others. I'm finding that I get the following problem when training using conjugate gradients: minimize: code= 13 nFE= 42 f= 487.91501 minimize: slope values inconsistent - function value is wobbly. I've tried changing Real to be a double rather than a float, but that didn't do the trick.. is there something else I can do? Thanks for any help. Mike Harm mharm@gizmo.usc.edu Cognitive Science / Psycholinguistics Lab Univ. of Southern California --------------------------------------------------------------- "I agree that 'two times two makes four' is an excellent thing; but if we are dispensing praise, then 'two times two makes five' is sometimes a most charming little thing as well." -Fyodor Dostoevsky, "Notes From Underground" From xerion-users-owner Tue Jan 19 19:04:09 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <150192>; Tue, 19 Jan 1993 19:04:00 -0500 Received: from localhost by neuron.ai.toronto.edu with SMTP id <244>; Tue, 19 Jan 1993 19:03:34 -0500 To: Mike Harm Subject: Re: wobbly function? In-reply-to: Your message of "Tue, 19 Jan 93 18:36:55 EST." <9301192336.AA06662@gizmo.usc.edu> cc: xerion-users@cs.toronto.edu Date: Tue, 19 Jan 1993 19:03:22 -0500 From: becker@ai.toronto.edu Message-Id: <93Jan19.190334edt.244@neuron.ai.toronto.edu> Mike, If you are randomly changing the frequency of training patterns during learning, the function value (total squared error) may go up or down independent of any change in the weights. That's why you are getting hte wobbly function message. So you can't use a line search or conjugate gradients. Try learning with steepest descent and momentum. Sue Becker From xerion-users-owner Thu Jan 28 12:13:50 1993 Received: from a.cs.okstate.edu ([139.78.9.1]) by relay.cs.toronto.edu with SMTP id <150181>; Thu, 28 Jan 1993 12:13:08 -0500 Date: Thu, 28 Jan 1993 12:08:47 -0500 From: KAVUTURU SIVA RAMA To: xerion-users@cs.toronto.edu Subject: Request for help to compile XERION on SEQUENT machine Message-Id: <93Jan28.121308est.150181@relay.cs.toronto.edu> Hai, I am trying to compile the XERION software on SEQUENT machine. I am getting some errors like strings.h not found stddef.h not found y.tab.h not found exampleparse.c not found .... Can you please help in solving this problem. I mailed many requests to xerion@ai.toronto.edu. But I did not get any reply. Please Help in solving this problem. Thank you very much. siva@a.cs.okstate.edu From xerion-users-owner Mon Feb 1 16:17:46 1993 Received: from sun2.nsfnet-relay.ac.uk ([128.86.8.45]) by relay.cs.toronto.edu with SMTP id <150199>; Mon, 1 Feb 1993 16:17:20 -0500 Via: uk.ac.oxford.prg; Mon, 1 Feb 1993 17:06:31 +0000 Received: from oxphys.physiol (oxphys-gate.physiol) by prg.oxford.ac.uk id AA05749; Mon, 1 Feb 93 16:15:05 GMT Received: from motor.physiol.physiol by uk.ac.oxford.physiology (4.1/physiol 1.2m (20-Jun-90)) id AA01828; Mon, 1 Feb 93 16:12:45 GMT Date: Mon, 1 Feb 1993 11:12:45 -0500 From: pem@physiology.oxford.ac.uk (Peter Menell) Message-Id: <9302011612.AA01828@uk.ac.oxford.physiology> To: xerion-users@ai.toronto.edu Subject: PC Has anyone out there ever ported Xerion to a PC or recompiled it on a PC running UNIX ?? cheers pete. From xerion-users-owner Tue Feb 16 16:00:12 1993 Received: from maestro.usc.edu ([128.125.111.1]) by relay.cs.toronto.edu with SMTP id <150201>; Tue, 16 Feb 1993 15:59:01 -0500 Received: by maestro.usc.edu (16.8/16.2) id AA04766; Tue, 16 Feb 93 12:59:57 -0800 From: Joseph Devlin Subject: Strange behavior To: xerion-users@cs.toronto.edu (Xerion User's List) Date: Tue, 16 Feb 1993 15:59:57 -0500 Mailer: Elm [revision: 70.30] Message-Id: <93Feb16.155901est.150201@relay.cs.toronto.edu> Hello all, I'm having a strange problem with presenting examples to a trained net in Xerion's bp module. Once training is complete and the net has basically learned the mapping I click on one of the examples in the Activations window to see what output activations it generates. Unfortunately clicking on the same example again (with no actions in between the first and second click) often changes the output activations. As I understand backprop an input produces a deterministic output if the weights aren't changing so this should not be happening. Furthermore these changes are often pretty radical - switching from one output to a completely different one - not a minor flux possibly created by a small change in weights. My question, then, is this: Is this a feature or a bug? Am I not understanding something that should be going on or did I make a mistake compiling this somewhere? (The latter is very possible as it took alot of changes to get Xerion up on my HP.) I've included a very small net as an example - I've run this example 20 times and 15 of those times produced these erroneous results although it apparently trains fine. Thanks for any help. - Joe email: jdevlin@maestro.usc.edu bogus.in: --------- addNet "Bogus" useNet "Bogus" addGroup -type INPUT Input 5 addGroup Hidden 3 addGroup -type OUTPUT Output 8 disconnectGroups Bias Input connectGroups Input Hidden connectGroups Hidden Output connectGroups Output Hidden orderGroups Bias Input Hidden Output randomize 1.0 0 doLayout bogus.layout addExamples -type TRAINING bogus.ex --------- bogus.ex: --------- 0 0 1 0 1, 1 1 0 1 0 0 0 1; 0 1 1 1 0, 0 1 1 0 0 1 0 0; 1 1 0 0 1, 0 0 0 0 1 1 0 1; 0 0 0 1 1, 1 0 0 1 0 1 0 1; 1 0 0 0 1, 0 1 0 0 1 0 1 1; 0 0 1 1 0, 1 1 1 1 0 0 0 0; ------------- bogus.layout: ------------- activity-layout cell-size 25 margin 3 per-row 10 all Output blank 1 all Input From xerion-users-owner Thu Feb 18 10:10:42 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <150223>; Thu, 18 Feb 1993 10:10:14 -0500 Received: by neuron.ai.toronto.edu id <602>; Thu, 18 Feb 1993 10:09:47 -0500 From: Drew van Camp To: jdevlin@maestro.usc.edu CC: xerion-users@cs.toronto.edu In-reply-to: Joseph Devlin's message of Tue, 16 Feb 1993 15:59:57 -0500 <93Feb16.155901est.150201@relay.cs.toronto.edu> Subject: Strange behavior Message-Id: <93Feb18.100947edt.602@neuron.ai.toronto.edu> Date: Thu, 18 Feb 1993 10:09:41 -0500 >From: Joseph Devlin >Date: Tue, 16 Feb 1993 15:59:57 -0500 >Mailer: Elm [revision: 70.30] > >Hello all, > I'm having a strange problem with presenting examples to >a trained net in Xerion's bp module. Once training is complete >and the net has basically learned the mapping I click on >one of the examples in the Activations window to see what >output activations it generates. Unfortunately clicking >on the same example again (with no actions in between the first >and second click) often changes the output activations. When I first read this, I thought there was a bug in the port to the HP, but when I had a look at the input file you posted, I realized that these are the expected results. >bogus.in: >--------- >addNet "Bogus" >useNet "Bogus" > >addGroup -type INPUT Input 5 >addGroup Hidden 3 >addGroup -type OUTPUT Output 8 > >disconnectGroups Bias Input >connectGroups Input Hidden >connectGroups Hidden Output >connectGroups Output Hidden ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >orderGroups Bias Input Hidden Output >randomize 1.0 0 >doLayout bogus.layout >addExamples -type TRAINING bogus.ex When you do a forward propagation the net will activate (in order) the groups: Bias, Input, Hidden, then Output. The way you've connected the net, the output activations are used as inputs for the hidden layer. This means that the hidden activations are calculated from the previous example's output activations (i.e. you're calculating the hidden unit activations before you calculate the output activations, so if the hidden layer tries to calculate its total input, it will use whatever the old output activations were). Basically, you've turned the bp net into a recurrent bp net. drew -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Mon Feb 22 10:25:57 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <150184>; Mon, 22 Feb 1993 10:24:15 -0500 Received: by neuron.ai.toronto.edu id <490>; Mon, 22 Feb 1993 10:23:58 -0500 From: Drew van Camp To: xerion-users@ai.toronto.edu Subject: Xerion Version 3.1 (and gcc) Message-Id: <93Feb22.102358edt.490@neuron.ai.toronto.edu> Date: Mon, 22 Feb 1993 10:23:51 -0500 Hi, As some of you may have noticed, I've put new versions of xerion up for ftp recently (version 3.1). I decided not to make an announcement about the release right away because I'll be making minor changes and bug fixes to the software over the next couple weeks. As I make the changes I'll put patches up for ftp and announce them. Once the software has stabilized, I'll freeze the release, and make a general announcement. There have been alot of improvements to xerion since version 3.0 (a complete list of which I'll be posting a list of bit later); however, I'm sure the improvements have introduced new bugs. The version up for ftp has been used in-house at U of T for several months now, with no real problems, so these bugs can't be too major. Overall, I would say that if you're using 3.0, you should consider upgrading to 3.1 some time soon. If you are really concerned about the stability of the software, then wait a couple of weeks (till the general announcement), and get it then. One of the bugs in the software right now has caused problems for people who compile with gcc. If you plan to get the code that's up for ftp right now, (or already have it), you should apply the patch I'm including below to the directory $XERIONDIR/src/etc before compiling. Thanks, drew ---------------------------- cut here ---------------------------- # # Apply this patch to create-bindings.awk. # # The error this patch fixes causes gcc to die when trying to # compile bindings.o for any simulator. # diff -c xerion.etc/create-bindings.awk:1.5 xerion.etc/create-bindings.awk:1.6 *** xerion.etc/create-bindings.awk:1.5 Mon Feb 22 09:46:16 1993 --- xerion.etc/create-bindings.awk Mon Feb 22 09:46:16 1993 *************** *** 1,5 **** ###################################################################### ! # $Id: create-bindings.awk,v 1.5 92/08/25 09:54:29 drew Exp $ ###################################################################### ###################################################################### --- 1,5 ---- ###################################################################### ! # $Id: create-bindings.awk,v 1.6 93/02/16 11:22:58 drew Exp $ ###################################################################### ###################################################################### *************** *** 45,51 **** basic_type["float"] = "float"; basic_type["String"] = "String"; basic_type["struct"] = "struct"; ! basic_type["union"] = "struct"; basic_type["LinkExtensionRec"] = "LinkExtensionRec" ; basic_type["NetExtensionRec"] = "NetExtensionRec" ; --- 45,51 ---- basic_type["float"] = "float"; basic_type["String"] = "String"; basic_type["struct"] = "struct"; ! basic_type["union"] = "union"; basic_type["LinkExtensionRec"] = "LinkExtensionRec" ; basic_type["NetExtensionRec"] = "NetExtensionRec" ; -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Tue Feb 23 05:24:56 1993 Received: from prip10.prip.tuwien.ac.at ([128.130.183.5]) by relay.cs.toronto.edu with SMTP id <150178>; Tue, 23 Feb 1993 05:24:05 -0500 Received: by prip10.prip.tuwien.ac.at (5.65/DEC-Ultrix/4.3) id AA01376; Tue, 23 Feb 1993 11:23:51 +0100 Message-Id: <9302231023.AA01376@prip10.prip.tuwien.ac.at> To: xerion-users@ai.toronto.edu Cc: bis@prip.tuwien.ac.at, stephan@prip.tuwien.ac.at From: bis@prip.tuwien.ac.at Subject: Help on Kohonen Date: Tue, 23 Feb 1993 12:23:51 -0500 Sender: bis@prip.tuwien.ac.at X-Mts: smtp Hi, We are using Xerion 3.1 and 3.0 and have problems with the Kohonen Simulator. Till now we were unable to use any own data in the example Networks (the network does simply not learn any neighbor relations), though we are able to run the sample simulations on Version 3.0. In Version 3.1 the sample networks also behave like standard Competitive Learning networks. Therfore my questions, has anyone experience with Kohnen networks. It would be very valuable if you could tell me how you managed to train the simulator. I have included the network definition we have used. Maybe we have made some stupid mistake. With best regards, Horst. -------------------------------------------------------------------------- addNet "Koho" useNet "Koho" addGroup -type INPUT Input 2 addGroup Output 25 disconnectGroups Bias Input disconnectGroups Bias Output connectGroups Input Output configureKohonenNet "Koho" 5 5 randomize 0.5 addExamples -t TRAINING DGP.data addExamples -t TESTING DGP.data doLayout koho.layout set currentNet.extension.initialMaxDistance = 4 set currentNet.extension.neighborhoodFunction = 1 set currentNet.extension.initialLearningRate = 0.001 set currentNet.extension.learningRateDecayFunction = 1 set currentNet.extension.learningRateDecay = 0.9999 set currentNet.mz.epsilon = 1 set currentNet.batchSize = 500 From xerion-users-owner Wed Feb 24 11:06:19 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <150186>; Wed, 24 Feb 1993 11:05:23 -0500 Received: by neuron.ai.toronto.edu id <511>; Wed, 24 Feb 1993 11:05:07 -0500 From: Drew van Camp To: xerion-users@ai.toronto.edu Subject: Xerion 3.1 Message-Id: <93Feb24.110507edt.511@neuron.ai.toronto.edu> Date: Wed, 24 Feb 1993 11:05:00 -0500 Hi folks, I've put the first patch for xerion V3.1 up for anonymous ftp in: ftp.cs.toronto.edu:/pub/xerion/xerion-3.1.patch1.Z This patch brings V3.1.111 up to V3.1.116. I've also changed the tar file for the library to reflect these changes (i.e., if you have an old version of xerion 3.1, the patch will bring it up to date with the version that is now up for ftp). The patch fixes the bug mentioned in my previous post, adds two new commands ('exec', and 'same'), as well as cleaning up the 'if...else' construct. I'm going to try to keep the ftp version in sync with the one we're using in-house for the next little while, so there should be patches once every week or two. Also, as promised, here is a list of the most important changes made between Versions 3.0 and 3.1 of the Xerion Neural Network Simulator. For a complete list of the changes see the file: ftp.cs.toronto.edu:/pub/xerion/CHANGES.ALL 1) A more general bp simulator, with more activation functions, error terms, cost terms, etc. 2) A new cascade correlation simulator (thanks to Brion Dolenko). 3) An improved and generalized method for handling the network complexity cost. 4) An improved command line interface. It now has conditional evaluations, loops, and command substitution. 5) Many new commands, among them: doExample - activate the network on a single example. doExamples - activate the network on an entire example set. calc - do floating point arithmetic calculations. exec - execute a xerion shell script. rangeTerminate - stop minimizing when all outputs are within some range of their targets 6) A new graphical interface for manipulating unit activation functions, error terms, network cost models, etc. once a network is built. 7) A new graphical interface for examining and setting general data structures. 8) An improved "Learning Methods" display. 9) A new display for showing a mixture of gaussian distribution (soft weight-sharing) cost model during training. 10) You can now show both incoming and outgoing connections in the "Connection Display". 11) The commands saveWeights and loadWeights have been greatly improved. 12) You can automatically save weights while training. 13) Examples have been modified to have: "tag" names; extensions; creation and destruction hooks; and to be able to store examples at arbitrary precision (down to 1 bit per input/target). 14) There are new methods for adding, copying, permuting, and deleting examples. 15) You can now constrain weights to be positive. 16) Xerion now uses a faster, table lookup sigmoid and a private (better) random number generator. 17) The code has been Standardized to make porting to other platforms easier. 18) *Loads* of minor changes and bug fixes. -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Wed Feb 24 14:09:12 1993 Received: from mercury.cs.uregina.ca ([142.3.200.53]) by relay.cs.toronto.edu with SMTP id <150192>; Wed, 24 Feb 1993 14:08:35 -0500 Received: by mercury.cs.uregina.ca id AA26945 (5.65c/IDA-1.4.4 for xerion-users@cs.toronto.edu); Wed, 24 Feb 1993 13:07:41 -0600 From: Jason Breti Message-Id: <199302241907.AA26945@mercury.cs.uregina.ca> Subject: Expert Systems & Xerion To: xerion-users@cs.toronto.edu Date: Wed, 24 Feb 1993 14:07:40 -0500 X-Mailer: ELM [version 2.3 PL11] Hi there! Has anyone done any work combining expert systems and Xerion? Any "Hands-On" implementations? If not, does anyone have any ideas on integrating an expert system with Xerion? Any help you can give would be much appreciated! I'm not currently a member of this list (working on that), so can you please respond directly to my userid? Thanks! -- Jason Breti jason@cs.uregina.ca Office: CL129.4 Phone: (306) 585-4975 Fax: -5205 regina!mercury!jason Department of Computer Science, University of Regina, Regina, Saskatchewan From xerion-users-owner Thu Mar 4 19:52:19 1993 Received: from hub.ubc.ca ([137.82.1.1]) by relay.cs.toronto.edu with SMTP id <150204>; Thu, 4 Mar 1993 19:50:48 -0500 Received: from org.ecc.ubc.ca by hub.ubc.ca (4.1/1.14) id AA19051; Thu, 4 Mar 93 16:50:28 PST Received: by org.ecc.ubc.ca (920330.SGI/1.14) id AA25462; Thu, 4 Mar 93 16:57:31 -0800 From: Zhengjin Shu Message-Id: <9303050057.AA25462@org.ecc.ubc.ca> Subject: Question Re: Example Sets To: xerion-users@ai.toronto.edu Date: Thu, 4 Mar 1993 19:57:30 -0500 X-Mailer: ELM [version 2.3 PL11] Hi, I am training a bp network with lots of examples. I am wondering in what sequence the examples in the training set are presented to the network during training, in the originally stored sequence, or in random order? In other words, if I want the training examples be presented in random order, do I have to randomize the examples in my .ex files, or the randomization is done after the examples are loaded into the network? Quick helps are very much appreciated. Thank you. _ _ - ( // .. ( / _///(//(///// )//// (// - Zhengjin Shu ( zhengjin@ee.ubc.ca ) From xerion-users-owner Fri Mar 5 09:36:35 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <150206>; Fri, 5 Mar 1993 09:36:01 -0500 Received: by neuron.ai.toronto.edu id <167>; Fri, 5 Mar 1993 09:35:47 -0500 From: Drew van Camp To: zhengjin@ecc.ubc.ca CC: xerion-users@cs.toronto.edu In-reply-to: Zhengjin Shu's message of Thu, 4 Mar 1993 19:57:30 -0500 <9303050057.AA25462@org.ecc.ubc.ca> Subject: Question Re: Example Sets Message-Id: <93Mar5.093547edt.167@neuron.ai.toronto.edu> Date: Fri, 5 Mar 1993 09:35:40 -0500 >I am training a bp network with lots of examples. I am wondering in what >sequence the examples in the training set are presented to the network during >training, in the originally stored sequence, or in random order? In other >words, if I want the training examples be presented in random order, >do I have to randomize the examples in my .ex files, or the >randomization is done after the examples are loaded into the network? The normal order of presentation is the order in which the examples are read in. If you want to randomize them after reading them in, use the command 'permuteExamples'. If you want them randomized after each pass through the set, set the field 'permute' in the exampleSet record: bp-> set currentNet.trainingExampleSet.permute = 1 drew -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Mon Apr 5 17:47:12 1993 Received: from SALK-SC2.SDSC.EDU ([192.31.153.12]) by relay.cs.toronto.edu with SMTP id <150187>; Mon, 5 Apr 1993 17:45:47 -0400 Received: from helmholtz.sdsc.edu by SALK-SC2.SDSC.EDU via SMTP with TCP; Tue, 30 Mar 93 16:58:34-PST Received: from pitts.sdsc.edu by helmholtz.sdsc.edu (4.1/SMI-4.1) id AA05090; Tue, 30 Mar 93 17:00:11 PST Date: Tue, 30 Mar 1993 20:00:11 -0500 From: magnus@helmholtz.sdsc.edu (Magnus Stensmo) Message-Id: <9303310100.AA05090@helmholtz.sdsc.edu> To: xerion-users@ai.toronto.edu Subject: Time Delay NNs Hello, I'd like to know if anybody has built time-delay back-propagation networks with Xerion 3.1 using the bp program. Is it possible? If so, how do you do it? The addExamples help page indicates a possibility and so does the addNet help page (but only for rbp). Thanks for any comments on this! Yours Magnus Stensmo -- Computational Neurobiology Laboratory (CNL) The Salk Institute P.O. Box 85800 San Diego, CA 92186-5800 U.S.A. From xerion-users-owner Tue Apr 20 21:29:31 1993 Received: from usc.edu ([128.125.253.136]) by relay.cs.toronto.edu with SMTP id <238221>; Tue, 20 Apr 1993 21:28:46 -0400 Received: from gizmo.usc.edu by usc.edu (4.1/SMI-3.0DEV3-USC+3.1) id AA10463; Tue, 20 Apr 93 18:28:34 PDT Received: by gizmo.usc.edu (4.1/SMI-4.1+ucs-3.6) id AA06148; Tue, 20 Apr 93 18:28:33 PDT From: mharm@gizmo.usc.edu (Mike Harm) Message-Id: <9304210128.AA06148@gizmo.usc.edu> Subject: lessening memory usage? To: xerion-users@ai.toronto.edu (xerion users) Date: Tue, 20 Apr 1993 21:28:33 -0400 X-Mailer: ELM [version 2.3 PL11] Hi folks. We are using xerion to run some fairly large networks and are running into memory problems. Does anyone have any experience with tuning or hacking up the xerion code to use less ram? For instance, can we safely wax the "name" field for units and links, by making it a null pointer or whatever? Are there any other tricks to lessen the expense of having lots of links? Any help would be greatly appreciated. cheers, -- Mike Harm mharm@gizmo.usc.edu Cognitive Science / Psycholinguistics Lab Univ. of Southern California --------------------------------------------------------------- "I agree that 'two times two makes four' is an excellent thing; but if we are dispensing praise, then 'two times two makes five' is sometimes a most charming little thing as well." -Fyodor Dostoevsky, "Notes From Underground" From xerion-users-owner Wed Apr 21 12:42:55 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <238225>; Wed, 21 Apr 1993 12:41:44 -0400 Received: by neuron.ai.toronto.edu id <738>; Wed, 21 Apr 1993 12:40:43 -0400 From: Drew van Camp To: mharm@gizmo.usc.edu CC: xerion-users@cs.toronto.edu In-reply-to: Mike Harm's message of Tue, 20 Apr 1993 21:28:33 -0400 <9304210128.AA06148@gizmo.usc.edu> Subject: lessening memory usage? Message-Id: <93Apr21.124043edt.738@neuron.ai.toronto.edu> Date: Wed, 21 Apr 1993 12:40:41 -0400 >From: mharm@gizmo.usc.edu (Mike Harm) >Date: Tue, 20 Apr 1993 21:28:33 -0400 >X-Mailer: ELM [version 2.3 PL11] >We are using xerion to run some fairly large networks and are running >into memory problems. Does anyone have any experience with >tuning or hacking up the xerion code to use less ram? For instance, >can we safely wax the "name" field for units and links, by making >it a null pointer or whatever? Are there any other tricks to lessen >the expense of having lots of links? Before you go trying to free the link names, there are a few things you can try: 1) Run xerion with no graphics when you have a really big network. The graphics can chew up a lot of memory: unix> bp -noGraphics 2) Weight constraints not only speed up the training of the network, but they decrease the amount of memory needed. If you can use them, do it. 3) Example sets can take up alot of memory too. There are two fields in the exampleSet structure: inputBits, and targetBits. They control how many bits of precision are used to store examples. If you have large examples with binary values, setting the fields to 1 can save *loads* of memory. You have to set these values before you load the examples: bp-> set currentNet.trainingExampleSet.inputBits = 1 bp-> set currentNet.trainingExampleSet.targetBits = 8 bp-> loadExamples -t TRAINING foo.ex If none of that works, you can start hacking at the code. 1) You can free the name of a unit then set it to NULL; however, I'd recommend setting it to the empty string "" just to be safe. (just make sure that if the name is not NULL, it can be de-allocated using "free"). 2) Memory management for links is done privately (because malloc can really thrash when you try to allocate 10000 links separately). The size of memory used for a link's name is a constant (32 bytes default). Even if a name is only three characters long, 32 bytes are allocated. If you want to change this default size you have to recompile '$XERIONDIR/src/lib/simulator/link.c" defining MAX_LINK_NAME to be the size you want. MAX_LINK_NAME *must* be at least 1. 3) Nets have arrays of groups and links; Groups have arrays of units; and units have arrays of links. All of these arrays are initially allocated a size GRANULARITY. When the array becomes full, the size is *doubled*. If you know a priori how big some of these arrays are going to be, you can redefine GRANULARITY in the appropriate files (in $XERIONDIR/src/lib/simulator/*.c). Try these things first, they should have the biggest effect on memory usage. drew -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Thu May 6 11:38:39 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <237411>; Thu, 6 May 1993 11:37:38 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <591>; Thu, 6 May 1993 11:37:26 -0400 To: xerion-users@ai.toronto.edu Subject: New patch for Xerion 3.1 Date: Thu, 6 May 1993 11:37:19 -0400 Sender: Drew van Camp From: Drew van Camp Message-Id: <93May6.113726edt.591@neuron.ai.toronto.edu> Hi everyone, I've put a new patch for Xerion Version 3.1 up for anonymous ftp in the file: ftp.cs.toronto.edu:/pub/xerion/xerion-3.1.patch2.Z This is the second patch, and it brings version 3.1.116 up to 3.1.147. I've also updated the tar files in the same directory so that they contain the latest release. Most of the changes in this patch have to do with bugs in, and modifications to, the command interface. There are essentially no changes to the 'simulator' part of Xerion. Here is the (almost) complete list of things that the patch fixes (or changes): 1) Various bug fixes in the installation procedure were fixed up. This should make it easier to compile for people running under openwindows. 2) There is a change in the variables the "exec" command creates and sets. Now it uses: ${tokc} - the number of command line tokens ${tokv[]} - the array of tokens (1, 2, ... tokc - 1) $* - the complete command line This syntax is much closer to that used when writing your own commands. 3) Variables can now be nested. E.g. bp-> var String foo bp-> var String bar bp-> set foo = bar bp-> set bar = junk bp-> echo ${foo} bar bp-> echo ${${foo}} junk 4) There is a new syntax for command substitution (back-quotes still work). It's a bit more powerful: bp-> echo `hello` Hello! bp-> echo $(hello) Hello! bp-> print $(unitLongName Output.0).output 0.9876 5) Aliases are now disabled by default (set "alias-enabled" to "true" to re-enable them). There is a deep bug in them that may mess up escaped characters. You can always use script files instead of aliases (they're more general, more powerful, and more debuggable). 6) The behaviour of "echo" and "read-echo" has been modified so that commands are echoed just before they're executed, and have been fully substituted. 7) There are two new commands for moving examples between example sets (moveExample and moveExamples). 8) There were various minor changes in the simulator library to improve performance. 9) There were various minor changes in the itf library to improve performance. This can make some script files (notably those that use loops and 'calc' alot) run about 10 times faster. 10) Minimize has been modified so that interrupts now cause it to register as having finished with an error (so that you can interrupt it when started from a script file). 11) Help message for addExamples are now up to date. 12) All BSD bcopy functions have been replaced with the equivalent memcpy functions. 13) Bug fixed where commands in loops and command substitutions were not being echoed. 14) A bug in fiddling weights in the connection display with the mouse has been fixed. 15) Fixed various bugs in the command line variable creation routines. 16) Fixed various bugs in the original hp port. 17) Fixed various bugs in caught only by gcc. As a side note: I get alot of questions about Xerion mailed to me directly, and I answer them as best as I can; but I really do encourage people to ask questions to this mailing list rather than to me. There are alot of people in our lab who are on the list and know as much about the simulator, and more about specific problems, than me. Also, seeing as traffic on this mailing list is so low, I'm going to start redistributing to it some of the more generally useful questions I get along with the answers. drew -- Drew van Camp ------------------------- xerion@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Wed May 12 10:07:30 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <237320>; Wed, 12 May 1993 10:06:43 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <574>; Wed, 12 May 1993 10:06:34 -0400 To: dsilver@csd.uwo.ca Cc: xerion-users@ai.toronto.edu Subject: Re: obtaining numerical values of output In-reply-to: Your message of "Tue, 11 May 93 16:14:34 EDT." <9305112014.AA25612@det.csd.uwo.ca> Date: Wed, 12 May 1993 10:06:27 -0400 Sender: Drew van Camp From: Drew van Camp Message-Id: <93May12.100634edt.574@neuron.ai.toronto.edu> In message <9305112014.AA25612@det.csd.uwo.ca> you wrote: > >Drew ... I was successful in installing Xerion on the faster box .. I thought >it was a MIPS but it turns out it was a Sparc-10. Regardless, your fix >worked! Currently, I am running 3.1.111. I guess I'll need to upgrade. It's probably a good idea to upgrade. I think some of the bugs that were fixed in the patch may affect the script I'm sending below (but try it first to see). >Now ... something new ... How do we get a dump of the networks output >(training, testing, validation) so that we can doa statistical comparison? It's quite easy to print the state of a unit for a specific example using the "doExamples" command and a trace. First make a file called dumpUnit (or whatever), that looks like the one below: ------------------------------ cut here ------------------------------ #!xerion # Create a variable that will hold the long name of a group (short # name is passed as an arg to the script) var String group set group = `groupLongName ${tokv[1]}` # on SYSV boxes (e.g. sgi, hp) use this to print output and target print ${group}.unit[*].output print ${group}.unit[*].target # on BSD boxes (e.g. sun), you have to use this more complicated command #show ${group}.unit | sed -n 's/^.*target = //p' | tr '\\012' ' ' ; echo '' #show ${group}.unit | sed -n 's/^.*output = //p' | tr '\\012' ' ' ; echo '' ------------------------------ cut here ------------------------------ When you type: bp-> dumpUnit Output # "> file.out" if you want to redirect the output this script will print all the activations of the units in "Output" on one line, and all the targets on the next line. To execute the script for all the examples in particular training sets just do the following: bp-> addTrace doExamples "dumpUnit Output" bp-> doExamples -t TRAINING bp-> doExamples -t TESTING That's all there is to it, drew >....Tx ... Danny >========================================================================= >= Daniel L. Silver University of Western Ontario, London, Canada = >= N6A 3K7 - Dept. of Comp. Sci. - Office: MC27b = >= dsilver@csd.uwo.ca H: (519)473-6168 O: (519)679-2111 (ext.6903) = >========================================================================= -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Thu May 13 12:48:50 1993 Received: from prip10.prip.tuwien.ac.at ([128.130.183.5]) by relay.cs.toronto.edu with SMTP id <237326>; Thu, 13 May 1993 12:47:53 -0400 Received: by prip10.prip.tuwien.ac.at (5.65/DEC-Ultrix/4.3) id AA21294; Thu, 13 May 1993 18:47:48 +0200 Message-Id: <9305131647.AA21294@prip10.prip.tuwien.ac.at> To: xerion-users@ai.toronto.edu Cc: bis@prip.tuwien.ac.at, ako@prip.tuwien.ac.at From: bis@prip.tuwien.ac.at Subject: Problems with DEC 5000/240 Date: Thu, 13 May 1993 19:47:48 -0400 Sender: bis@prip.tuwien.ac.at X-Mts: smtp Hi ! We are using Xerion 3.1 on Sun Workstations. Everything works fine. Recently we have tried to install the simulator on a DEC 5000/240. We had some problems in compiling (especially with gcc). But after defining HZ in itf we got it to work. So now the simulations work fine. However if you have trained a network and you want to save the weights you get an error: Command error: saveWeights: error writing "...." Has anyone had similar problems? How can we solve them ? Any help you can give would be much appreciated! Horst ------------------------------------------------------------------------ Horst Bischof bis@prip.tuwien.ac.at Inst. f. Automation Dept. f. Pattern Recognition and Image Processing Technical University Vienna Treitlstr. 3/1832 A-1040 Vienna From xerion-users-owner Fri May 14 10:02:39 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <237344>; Fri, 14 May 1993 10:01:35 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <753>; Fri, 14 May 1993 10:01:29 -0400 To: bis@prip.tuwien.ac.at Cc: xerion-users@ai.toronto.edu Subject: Re: Problems with DEC 5000/240 In-reply-to: Your message of "Thu, 13 May 93 19:47:48 EDT." <9305131647.AA21294@prip10.prip.tuwien.ac.at> Date: Fri, 14 May 1993 10:01:17 -0400 Sender: Drew van Camp From: Drew van Camp Message-Id: <93May14.100129edt.753@neuron.ai.toronto.edu> In message <9305131647.AA21294@prip10.prip.tuwien.ac.at> you wrote: >Hi ! > >We are using Xerion 3.1 on Sun Workstations. Everything >works fine. >Recently we have tried to install the simulator on >a DEC 5000/240. We had some problems in compiling >(especially with gcc). But after defining HZ in itf >we got it to work. >So now the simulations work fine. However if you >have trained a network and you want to save the >weights you get an error: > >Command error: saveWeights: error writing "...." > >Has anyone had similar problems? >How can we solve them ? >Any help you can give would be much appreciated! > Hi, I've heard about this problem under ultrix before. I don't have access to any DEC machines where I am, and the people who do haven't been able to track the problem down. However, all is not lost. You can save the weights using the 'show -set' command: bp-> show -set currentNet.links > foo.wt Or if you want to be fancy (and have a smaller weight file): bp-> show -set currentNet.links | grep 'weight =' > foo.wt Then when you want to reload the weights, just type: bp-> read foo.wt >Horst >------------------------------------------------------------------------ >Horst Bischof bis@prip.tuwien.ac.at >Inst. f. Automation >Dept. f. Pattern Recognition and Image Processing >Technical University Vienna >Treitlstr. 3/1832 A-1040 Vienna drew -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Fri May 14 14:58:23 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <237330>; Fri, 14 May 1993 14:57:23 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <119>; Fri, 14 May 1993 14:57:07 -0400 To: dsilver@csd.uwo.ca Cc: xerion-users@ai.toronto.edu Subject: Re: Got another one for you ... In-reply-to: Your message of "Fri, 14 May 93 14:09:01 EDT." <9305141809.AA12687@no8sun.csd.uwo.ca> Date: Fri, 14 May 1993 14:56:51 -0400 Sender: Drew van Camp From: Drew van Camp Message-Id: <93May14.145707edt.119@neuron.ai.toronto.edu> In message <9305141809.AA12687@no8sun.csd.uwo.ca> you wrote: >I have been trying to use the format command to set format of the >network target and actual output values to be 0.999999 ("%.6g"). >So far no success .. I keep getting an error message saying the input >must be in proper structure format. I have tried everything I can >think of. Specifically we want to output the following: >currentNet.group[3].unit.output >currentNet.group[3].unit.target Formats work on *types* of structures, not on instances of them. The upshot of this is that if you try "whatis target" you'll see: bp-> whatis target "target" is a field of type "Real" in the structure "UnitRec" Which you can then use for: bp-> format UnitRec.target "%.6g" Which should have the desired result. >.... Thanks again ... Danny >========================================================================= >= Daniel L. Silver University of Western Ontario, London, Canada = >= N6A 3K7 - Dept. of Comp. Sci. - Office: MC27b = >= dsilver@csd.uwo.ca H: (519)473-6168 O: (519)679-2111 (ext.6903) = >========================================================================= drew -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Tue May 18 07:04:45 1993 Received: from ugw.utcc.utoronto.ca ([128.100.100.3]) by relay.cs.toronto.edu with SMTP id <237384>; Tue, 18 May 1993 07:03:04 -0400 Received: from vec.ccupm.upm.es by ugw.utcc.utoronto.ca with BSMTP id <9032>; Tue, 18 May 1993 07:02:36 -0400 Received: from ccupm.upm.es (C0505002) by vec.ccupm.upm.es (Mailer R2.08) with BSMTP id 6949; Tue, 18 May 93 12:21:11 GMT Date: Tue, 18 May 1993 08:20:03 -0400 From: Pedro Mejia Organization: Universidad Politecnica de Madrid (Spain) Subject: Problems with Instalation To: xerion-users@ai.toronto.edu Message-Id: <93May18.070236edt.9032@ugw.utcc.utoronto.ca> ======================================================================== 47 Dear Sirs: I have retrieve your Xerion Neural Network Simulator, but I have some problem s with its instalation. I am running your software on a Sun Spark Station with Ver. Sun O.S 4.1.2. In your README file there is a Note: ''Xerion uses Imakefiles; therefore, you must have imake and xmkmf installed on your system to build it'' Are these two files, the only one needed ??, or what other make files are ne cesary. When I run '' Make World '', abort its execution because, reports that it can 't find a Makedirhier file on /usr/bin/X11 directory. But I have not such directory, and Makedirhier is on /usr/X11/bin. Also the Makefiles have some other 'paths' diferent than mine, so please tell me how can I change them and on wich files. I have made a 'trick' creating a /usr/X11/bin directory with the Makedirhier file, and it passes this point, but also I have and abort on the: make includes make depend and make all Also, becouse it have, on its instalation files, a diferent paths than those on my directories structure. Please tell me how can I fix these problems, with the instalation of your Sof tware. If there is any sugestion or additional comment, I would appreciate it v ery much. Thank's. Best Regards. Pedro Mejia University of Madrid, SPAIN From xerion-users-owner Thu May 20 15:03:55 1993 Received: from SALK-SCI.SDSC.EDU ([192.31.153.10]) by relay.cs.toronto.edu with SMTP id <237319>; Thu, 20 May 1993 14:59:12 -0400 Received: from helmholtz.sdsc.edu by SALK-SCI.SDSC.EDU via SMTP with TCP; Thu, 20 May 93 11:56:39-PST Received: from fatt by helmholtz.sdsc.edu (4.1/SMI-4.1) id AA17024; Thu, 20 May 93 12:01:00 PDT Received: by fatt (5.61/1.34) id AA02324; Thu, 20 May 93 12:00:58 -0700 Message-Id: <9305201900.AA02324@fatt> To: xerion-users@cs.toronto.edu Subject: Ctrl-C in an inner minimization Date: Thu, 20 May 1993 15:00:58 -0400 From: "Richard S. Zemel" I have added an inner minimization to my simulator, and was wondering how I can make ctrl-C behave as it does during the standard minimize. As it is now, the simulation is just interrupted and the network can be left in a funny state. I like the levels of interrupt that the standard minimization has. Thanks for any help, Rich From xerion-users-owner Thu May 20 15:48:51 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <237312>; Thu, 20 May 1993 15:48:23 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <490>; Thu, 20 May 1993 15:48:12 -0400 To: xerion-users@ai.toronto.edu, zemel@helmholtz.sdsc.edu Subject: Re: Ctrl-C in an inner minimization In-reply-to: Your message of "Thu, 20 May 93 15:00:58 EDT." <9305201900.AA02324@fatt> Date: Thu, 20 May 1993 15:48:00 -0400 Sender: Tony Plate From: Tony Plate Message-Id: <93May20.154812edt.490@neuron.ai.toronto.edu> I don't quite understand how you've got the inner minimization set up, so I don't know exactly what to suggest, but here's how minimize handles Ctrl-C, maybe you can work things out from this. The minimize command installs the function interruptMinimize() as the SIGINT signal handler. This function prints a message about how many times it has been called, and if it is less than 3 times it increments a "stop flag". If it is 3 or more times it aborts immediately (leaving the network in an unknown state). The minimize loop checks the stop flag at two locations: at the end of a step, and after each network evaluation. It exits at the first location if the stop flag >=1, and at the second location if the stop flag is >=2, (in either case leaving the network in a tidy state). You could install this function or a similar one to handle your inner loop, but if your inner loop returns to minimize you should be careful to restore the signal handler to what it was upon entering the inner loop. If your inner loop just returns to the command interface the interface will take care of restoring the signal handler. That function is both defined and used in xerion/src/lib/minimize.c. Tony Plate From xerion-users-owner Tue May 25 10:43:54 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.cs.toronto.edu with SMTP id <238256>; Tue, 25 May 1993 10:42:11 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <334>; Tue, 25 May 1993 10:42:02 -0400 To: "Rajeev K. Saraf" Cc: xerion-users@ai.toronto.edu Subject: Re: Adding a new command to xerion In-reply-to: Your message of "Wed, 19 May 93 19:36:22 -0400." <9305192336.AA28029@necs.vuse> Date: Tue, 25 May 1993 10:41:47 -0400 Sender: Drew van Camp From: Drew van Camp Message-Id: <93May25.104202edt.334@neuron.ai.toronto.edu> In message <9305192336.AA28029@necs.vuse> you wrote: >Hi Drew, > > Thank you for distributing the XERION software free and I appreciate >the effort put by your group in developing it. I have been using it >for a while and i really like it. Recently, while working on the >experiments, i had to add a new command to xerion command list. I >read thru the manual and the documenation and still was not very >clear on doing it. > >I created the command ( *.c file) file in the ./src/lib/commands >directory and compiled it. then i ran the extract-commands script. I >have not been able to figure out how to incorporate the command into >the command list. I have tried all possibilities i could think of >but in vain. I would highly appreciate if you can send me detail >documenatation or step by step procedure for including a new command. Ok, here's the easiest way to create a new command: 1) Normally you add new commands to a specific simulator (not the libraries) for example the bp simulator, so change into the directory containing the source for the bp simulator: unix> cd $XERIONDIR/src/sim/bp # or wherever you keep the source 2) There is a template file for commands in $XERIONDIR/templates, so copy it into the working directory (make sure you rename it, since command.c already exists there): unix> cp $XERIONDIR/templates/command.c newcommand.c 3) Edit the line 'OBJS =' in the Imakefile, adding the name 'newcommand.o' 17c17 < OBJS = bp.o unit.o command.o --- > OBJS = bp.o unit.o command.o newcommand.o 4) Now rebuild the makefile. unix> make Makefile unix> make depend 5) Now make normally, and you'll have a new command called 'template' that you can rename, and modify as you want (recompiling between changes of course). The commands are automatically extracted (using the script you mentioned), so you don't have to worry about it. unix> make # followed by 'make install' if you plan to install it >Thnaking in advance > >Rajeev Saraf >sarafrk@vuse.vanderbilt.edu > drew -- Drew van Camp ------------------------- xerion@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Thu Jun 3 17:04:06 1993 Received: from maestro.usc.edu ([128.125.111.1]) by relay.cs.toronto.edu with SMTP id <237377>; Thu, 3 Jun 1993 17:00:59 -0400 Received: by maestro.usc.edu (16.8/16.2) id AA03217; Thu, 3 Jun 93 13:59:14 -0700 From: Joseph Devlin Subject: The show and set commands To: xerion-users@cs.toronto.edu (Xerion User's List) Date: Thu, 3 Jun 1993 16:59:14 -0400 Mailer: Elm [revision: 70.30] Message-Id: <93Jun3.170059edt.237377@relay.cs.toronto.edu> Hi Folks, I was wondering whether anyone had hacked through the "show" or "set" command in Xerion to determine how they work? These are pretty sophisticated commands, if you think about it, since both access the internals of the program itself in some way - there is no code that explicitly handles each and every field of all the xerion variables. For instance, if one adds an extension record like the following: typedef struct NetExtensionRec { /* BIND */ Real bogusValue ; /* My new field */ } NetExtensionRec ; Then within Xerion you can "show currentNet.extension.bogusValue" without having changed any code related to the show command. In other words, it is somehow examining its own structures and determining what value to display. "set" does the same thing. I haven't figured out how this works yet so I thought that if anyone else had maybe they'd be willing to share the fruits of their labors? Thanks much! - Joe From xerion-users-owner Thu Jun 3 17:20:00 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <237376>; Thu, 3 Jun 1993 17:19:31 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <821>; Thu, 3 Jun 1993 17:19:24 -0400 To: xerion-users@cs.toronto.edu, Joseph Devlin Subject: Re: The show and set commands In-reply-to: Your message of "Thu, 03 Jun 93 16:59:14 EDT." <93Jun3.170059edt.237377@relay.cs.toronto.edu> Date: Thu, 3 Jun 1993 17:19:17 -0400 Sender: Tony Plate From: Tony Plate Message-Id: <93Jun3.171924edt.821@neuron.ai.toronto.edu> Joseph Devlin writes: > I was wondering whether anyone had hacked through the > "show" or "set" command in Xerion to determine how they work? Well, I hacked through them once, when I wrote them :-) Xerion knows the offsets and types of all the fields that show and set can manipulate. The .bindings.c file is where that information is computed. For each field in each structure, there is a function call to enter that field in a table, along with its offset and type. The offset is calcalated by the compiler. Table entries are also created for each structure and variables that have the BIND keyword in a comment on the same line. The .bindings.c file is created by an awk program, which is an extremely ugly mess (or it was when I left it with Drew, I'm not sure if he's cleaned it up). This awk program does very a very dumb parse of variable and typedef declarations, it is easily confused by strange formatting, but most people format their typedef declarations in the same one (one field per line), so it works. If you want to find out more, you can look at the .bindings.c file to start with. Take your example: typedef struct NetExtensionRec { /* BIND */ Real bogusValue ; /* My new field */ } NetExtensionRec ; here's the relevant code put in bindings.c: struct NetExtensionRec *NetExtensionRec_ptr = NULL; IEnterType("NetExtensionRec",sizeof(struct NetExtensionRec),IS_STRUCT); IEnterField("NetExtensionRec", "Real", "bogusValue",(char*)&(NetExtensionRec_ptr->bogusValue)-(char*)NetExtensionRec_ptr,P_OBJECT, 0, ""); - Tony Plate From xerion-users-owner Fri Jun 4 09:54:52 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <237350>; Fri, 4 Jun 1993 09:54:12 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <768>; Fri, 4 Jun 1993 09:54:02 -0400 To: Tony Plate cc: xerion-users@cs.toronto.edu, Joseph Devlin Subject: Re: The show and set commands In-reply-to: Your message of "Thu, 03 Jun 93 17:19:17 EDT." <93Jun3.171924edt.821@neuron.ai.toronto.edu> Date: Fri, 4 Jun 1993 09:53:50 -0400 Sender: Drew van Camp From: Drew van Camp Message-Id: <93Jun4.095402edt.768@neuron.ai.toronto.edu> In message <93Jun3.171924edt.821@neuron.ai.toronto.edu> you wrote: > >Joseph Devlin writes: >> I was wondering whether anyone had hacked through the >> "show" or "set" command in Xerion to determine how they work? > >Well, I hacked through them once, when I wrote them :-) >Xerion knows the offsets and types of all the fields that >show and set can manipulate. The .bindings.c file is ^^^^^^^^^^^ >where that information is computed. For each field in >each structure, there is a function call to enter that >field in a table, along with its offset and type. The >offset is calcalated by the compiler. Actually, the only noticeable change I made to the bindings stuff is renaming this file "bindings.c". "make" deletes the file once it's finished compiling it, so it's rather hard to look at it. You can edit the file $XERIONDIR/config/xerion.rules so that it doesn't delete it (if you really want to). >- Tony Plate drew -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Mon Jun 14 05:08:46 1993 Received: from thrall.cm.cf.ac.uk ([131.251.42.22]) by relay.cs.toronto.edu with SMTP id <237343>; Mon, 14 Jun 1993 05:00:22 -0400 Received: from cm.cf.ac.uk by thrall.cm.cf.ac.uk with SMTP (PP) id <05820-0@thrall.cm.cf.ac.uk>; Mon, 14 Jun 1993 10:03:16 +0100 Date: Mon, 14 Jun 1993 05:00:05 -0400 From: Daniel K R McCormack Message-Id: <9306140900.AA19691@heliodor.cm.cf.ac.uk> To: xerion-users@cs.toronto.edu Subject: Rules of thumb for weight sharing Hi, I was wondering if anyone could suggest any rules of thumb for setting up weight sharing networks. I am trying to recognise an image 128 x 64 pixels and I am not sure how to go about deciding the window sizes , number of layers and connectivity etc etc... to use for best results. Also - Is it viable to break the image up into say 32 x 32 pixel blocks, and use weight sharing on these sub windows inorder to have some kind of limit to the position invariance achived by weight sharing ? ------------------------------------------- As a quick aside question Is epsilon in bp part of xerion equivalent to eta in the standard BP update? In other words is minimize -epsilon 0.01 -momentum 0.9 equivalent to standard BP with eta = 0.01 and momentum = 0.9 or do I have to do anything else ??? Many Thanks Daniel McCormack From xerion-users-owner Mon Jun 28 22:57:02 1993 Received: from uqcspe.cs.uq.oz.au ([130.102.192.8]) by relay.cs.toronto.edu with SMTP id <80367>; Mon, 28 Jun 1993 22:55:40 -0400 Received: from pansy.cs.uq.oz.au by uqcspe.cs.uq.oz.au id ; Tue, 29 Jun 93 12:55:23 +1000 Date: Mon, 28 Jun 1993 22:55:22 -0400 From: bakker@cs.uq.oz.au Message-Id: <9306290255.AA21381@client> To: xerion-users@cs.toronto.edu Xerion Users, I seem to be experiencing a problem with the batchSize option in Xerion (bp). I have a training set of 146 patterns, and I am using steepest descent with no momentum as the training algorithm - the standard setup. I have 189 input units, 10 hidden units, and 26 output units. The default batchsize is "all", but I set it to 1. My intention is that each pattern in the training set be presented in turn, and the weights be updated immediately after each single pattern presentation. However, I have found that the network's output freezes to one combination of output values after 30 or so epochs. It produces the *same* output for every training pattern. And this output is the *target* for the first pattern in the training set. When I change the first pattern in the training set, the behaviour is the same - it learns the first target only. It would appear that when you set a batchSize of 1, the network only trains on the first pattern over and over again. My question is: What am I doing wrong? How do I get Xerion(bp) to do on-line (pattern-update) training? We use Xerion, V3.1.147. Paul Bakker bakker@cs.uq.oz.au From xerion-users-owner Tue Jun 29 09:41:43 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <452091>; Tue, 29 Jun 1993 09:35:43 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <214>; Tue, 29 Jun 1993 09:35:30 -0400 To: bakker@cs.uq.oz.au Cc: xerion-users@ai.toronto.edu In-reply-to: Your message of "Mon, 28 Jun 93 22:55:22 EDT." <9306290255.AA21381@client> Date: Tue, 29 Jun 1993 09:35:26 -0400 Sender: Drew van Camp From: Drew van Camp Message-Id: <93Jun29.093530edt.214@neuron.ai.toronto.edu> Hi, It sounds like you're doing everything correctly for online learning. I just tried the same thing for a simple network on both an SGI and Sun Sparc using the same version of Xerion as you. I got the expected result (eventually it learned all the patterns). Is it possible that you have some trace command that is resetting the example set's currentIdx after each iteration (which for online learning is after each pattern presentation)? That would cause it to only show the first example during training. "doExamples" is a good example of this type of command. Try deleting all traces and then training, that might solve the problem. drew -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Tue Jun 29 20:38:08 1993 Received: from uqcspe.cs.uq.oz.au ([130.102.192.8]) by relay.cs.toronto.edu with SMTP id <452099>; Tue, 29 Jun 1993 20:37:03 -0400 Received: from pansy.cs.uq.oz.au by uqcspe.cs.uq.oz.au id ; Wed, 30 Jun 93 10:36:48 +1000 Date: Tue, 29 Jun 1993 20:36:47 -0400 From: bakker@cs.uq.oz.au Message-Id: <9306300036.AA23376@client> To: xerion-users@cs.toronto.edu Subject: On-line learning problem Drew writes: >Is it possible that you have some trace command that is resetting the >example set's currentIdx after each iteration I've located the problem. I was using the 0.5 pattern criterion, ie I wanted training to stop if the error on every output for every pattern was less than 0.5. The command is: addTrace minimizeEndRepetition "rangeTerminate -p 0.5" When I removed this command, it worked fine. Quite a surprising side-effect, but I guess the rangeTerminate command is incompatible with on-line learning. paul bakker bakker@cs.uq.oz.au From xerion-users-owner Wed Jun 30 09:55:49 1993 Received: from neuron.ai.toronto.edu ([128.100.3.14]) by relay.cs.toronto.edu with SMTP id <452100>; Wed, 30 Jun 1993 09:39:14 -0400 Received: from localhost by neuron.ai.toronto.edu with SMTP id <210>; Wed, 30 Jun 1993 09:38:58 -0400 To: bakker@cs.uq.oz.au cc: xerion-users@cs.toronto.edu Subject: Re: On-line learning problem In-reply-to: Your message of "Tue, 29 Jun 93 20:36:47 EDT." <9306300036.AA23376@client> Date: Wed, 30 Jun 1993 09:38:43 -0400 Sender: Drew van Camp From: Drew van Camp Message-Id: <93Jun30.093858edt.210@neuron.ai.toronto.edu> In message <9306300036.AA23376@client> you wrote: > >Drew writes: >>Is it possible that you have some trace command that is resetting the >>example set's currentIdx after each iteration > >I've located the problem. I was using the 0.5 pattern criterion, ie I wanted >training to stop if the error on every output for every pattern was less than >0.5. The command is: > >addTrace minimizeEndRepetition "rangeTerminate -p 0.5" > >When I removed this command, it worked fine. >Quite a surprising side-effect, but I guess the rangeTerminate command >is incompatible with on-line learning. There is a way to still use rangeTerminate. You just have to be sure that after you call it you reset the example set's currentIdx to the proper value. The following will do exactly that: --------------------------- cut here ------------------------------- # create a variable to save the correct currentIdx var int exampleIdx # First save the currentIdx addTrace minimizeEndRepetition \ 'set exampleIdx = ${currentNet.trainingExampleSet.currentIdx}' # Then do the rangeTerminate addTrace minimizeEndRepetition 'rangeTerminate -p 0.5' # Finally restore the currentIdx addTrace minimizeEndRepetition \ 'set currentNet.trainingExampleSet.currentIdx = $exampleIdx' --------------------------- cut here ------------------------------- >paul bakker >bakker@cs.uq.oz.au drew -- Drew van Camp --------------------------- drew@cs.toronto.edu -- -- Dept. of Computer Science, University of Toronto --------------- -- 6 Kings College Road, Toronto, Ontario -- Vox: (416) 978-7403 -- -- CANADA M5S 1A4 -------------------------- Fax: (416) 978-1455 -- From xerion-users-owner Wed Jul 7 13:45:37 1993 Received: from p.lanl.gov ([128.165.4.4]) by relay.cs.toronto.edu with SMTP id <452106>; Wed, 7 Jul 1993 13:42:52 -0400 Received: from c3.lanl.gov by p.lanl.gov (5.65/1.14) id AA11743; Wed, 7 Jul 93 10:02:05 -0600 Received: from c3serve.c3.lanl.gov (c3serve-e1.c3.lanl.gov) by c3.lanl.gov (4.1/5.28) id AA17520; Wed, 7 Jul 93 10:01:53 MDT Received: from father.c3.lanl.gov by c3serve.c3.lanl.gov (4.1/SMI-4.0) id AA23670; Wed, 7 Jul 93 10:01:53 MDT Date: Wed, 7 Jul 1993 12:01:53 -0400 From: crabbe@c3serve.c3.lanl.gov (Ric Crabbe) Message-Id: <9307071601.AA23670@c3serve.c3.lanl.gov> To: xerion-users@ai.toronto.edu Subject: xerion build problems: sun 4.1.3 Cc: dnix@c3.LANL.GOV, crabbe@c3.LANL.GOV We can't seem to get xerion to compile here on our sun4s : SunOS Release 4.1.3 everything seems to be going fine until suddenly: acc -o kick kick.o -O -pipe -L/usr/lib/X11 install -c kick /tmp_mnt/n/c3serveg/crabbe/simulator/xerion/bin/sparc acc -O -pipe -I../.././src/lib/include -D_BSD_COMPAT -D_NO_LIBPW -target sun4 -c xkick.c "../.././src/lib/include/xerion/config.sun.h", line 39: identifier redeclared: atof "../.././src/lib/include/xerion/config.sun.h", line 40: identifier redeclared: strtod "/usr/lang/SC1.0/ansi_include/stdio.h", line 136: warning: identifier redeclared: sprintf Compilation failed *** Error code 2 make: Fatal error: Command failed for target `xkick.o' Current working directory /tmp_mnt/n/c3serveg/crabbe/simulator/xerion/src/util *** Error code 1 make: Fatal error: Command failed for target `all' Current working directory /tmp_mnt/n/c3serveg/crabbe/simulator/xerion/src *** Error code 1 make: Fatal error: Command failed for target `all' Current working directory /tmp_mnt/n/c3serveg/crabbe/simulator/xerion *** Error code 1 make: Fatal error: Command failed for target `World' Thus ending the attempt. The only other suspicous output is: makedepend -s "# DO NOT DELETE" -- -I../../.././src/lib/include -D_BSD_COMPAT -D_NO_LIBPW -- alias.c arg-actual.c arg-desc.c arg-print.c arg-store.c background.c bind-enter.c bind-print.c bind-var.c calc.c hash-table.c itf-argf.c itf.c lex.c loop.c mem.c option.c range.c stack.c stream.c trace.c utils.c varsub.c version.c warn.c makedepend: cannot open "calc.c" We've tried commenting out offending lines in config.sun.h but that creates an epic mess. has anyone compiled xerion on 4.1.3 and can give us some tips on where to go from here? Thanks, ric -- Ric Crabbe Staff Research Assistant Los Alamos National Laboratory crabbe@c3.lanl.gov From xerion-users-owner Wed Jul 7 14:05:44 1993 Received: from marlowe.cog.brown.edu ([128.148.208.11]) by relay.cs.toronto.edu with SMTP id <452112>; Wed, 7 Jul 1993 14:04:48 -0400 Received: by marlowe.cog.brown.edu (5.57/Ultrix3.0-C) id AA11060; Wed, 7 Jul 93 14:02:45 -0400 Message-Id: <9307071802.AA11060@marlowe.cog.brown.edu> Subject: backprop through time To: xerion-users@cs.toronto.edu (Xerion users) Date: Wed, 7 Jul 1993 14:02:44 -0400 From: David_Ascher@brown.edu (David Ascher) Reply-To: David_Ascher@brown.edu (David Ascher) X-Mailer: ELM [version 2.3 PL11] What's the best way to get backprop through time working with Xerion? Anyone have a bptt module out there? Thanks for any info. David Ascher Brown University Box 1978 Providence RI 02912 Department of Cognitive and Linguistic Sciences David_Ascher@Brown.EDU (401) 863-1167 From xerion-users-owner Fri Jul 9 07:50:42 1993 Received: from rulway.LeidenUniv.nl ([132.229.8.6]) by relay.cs.toronto.edu with SMTP id <452154>; Fri, 9 Jul 1993 07:48:41 -0400 Received: from hp3.wi.leidenuniv.nl by rulway.LeidenUniv.nl with SMTP id AA18845 (5.65c+/IDA-1.4.4 for ); Fri, 9 Jul 1993 13:48:35 +0200 Message-Id: <199307091148.AA18845@rulway.LeidenUniv.nl> Received: by hp3.wi.leidenuniv.nl (16.8/16.2) id AA18252; Fri, 9 Jul 93 13:46:55 +0200 From: Ro Bloem Subject: Xerion on HP - replacement for 'test' To: xerion-users@ai.toronto.edu Date: Fri, 9 Jul 1993 07:46:55 -0400 Mailer: Elm [revision: 70.30] Hello, I'm installing Xerion on a HP-720. The HP does not have a test command in it's c-shell. Yet config/setup.csh uses test in it's first line. How can I alter setup.csh (or the HP) so that it will run, without changing shells? Thanks for helping. Roderick Bloem dept. of mathematics and computerscience, university of Leiden, the Netherlands.