Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!rochester!cornellcs!newsstand.cit.cornell.edu!newstand.syr.edu!news.maxwell.syr.edu!cpk-news-hub1.bbnplanet.com!cam-news-hub1.bbnplanet.com!news.bbnplanet.com!howland.erols.net!rill.news.pipex.net!pipex!uknet!usenet1.news.uk.psi.net!uknet!uknet!bcc.ac.uk!news
From: Stevie Herbert Sackin <stevie@psychol.ucl.ac.uk>
Subject: BPTT
Message-ID: <330C2FE0.18E4@psychol.ucl.ac.uk>
Date: Thu, 20 Feb 1997 11:05:04 GMT
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0
X-Mailer: Mozilla 3.0 (X11; I; SunOS 5.3 sun4m)
Organization: University College London
Lines: 31

Hello,

I'm trying to implement a recurrent ANN (as described by 
Werbos (1990)), using a BPTT learning algorithm (batch 
processing, tanh transfer function and delta-bar-delta 
weight update). I have looked at some of the ANN simulators 
but thought it would be quicker to write the code from 
scratch for my application (ha ha ha!).

Having now finished the program (C) I'm attempting to 
test/debug it and have encountered some weird stuff 
happening with the learning curves. They usually start off 
looking OK then go into sections of 'instability' (shooting up and
down all over the place). The curves are particularly poor
when the ANN learns the patterns correctly (which only occurs
about 50% of the time).

The patterns I'm using are xor patterns with 
a twist - '0 0' is classified as 0 if preceded by '1 1'
or '0 0' and 1 if preceded by '1 0' or '0 1'.

If anyone has any ideas about what I may be doing wrong and/or
some benchmark data which has been well documented for this
kind of algorithm and/or some code which I could test against my
own I would most greatly appreciate hearing from you.

Yours

stevie sackin
PS I get similar learning curves with steepest descent weight 
update

