Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!rochester!cornellcs!newsstand.cit.cornell.edu!news.kei.com!newsfeed.internetmci.com!in1.uu.net!halon!sybase!usenet
From: George Van Treeck <treeck>
Subject: Re: Newbie question - why some patterns are not learned?
Content-Type: text/plain; charset=us-ascii
Message-ID: <DI0649.KKC@sybase.com>
Sender: usenet@sybase.com
Content-Transfer-Encoding: 7bit
Organization: Sybase, Inc.
References: <1995Nov13.121929.37328@ucl.ac.uk>
Mime-Version: 1.0
Date: Mon, 13 Nov 1995 22:21:44 GMT
X-Mailer: Mozilla 1.1N (X11; I; SunOS 5.3 sun4m)
X-Url: news:1995Nov13.121929.37328@ucl.ac.uk
Lines: 24

>I have just one quick question. Why are some patterns learned by a
>network very quickly whereas others seem to take forever? I currently
>have 11 input units, 200 hidden and 150 outputs and am using backprop
>to train on 13! abitrary vector pairs. After 50 epochs  9 of the 13
>have been learned. After 3000 epochs, the remaining 4 are stubbornly
>refusing to be learnt. What constraints are there on learnability?

It could be that you over trained on the other patterns -- making it
difficult to learn very different patterns.  It could be the four patterns
are very similar to other already learned patterns, but you are asking for
a response is very different than the other patterns (interference).
It could be a flaw in the learning algorithms. Maybe it just take
10,000 epochs for that particular data set/learning algorithms.
And so on -- many possible causes.

Something to try:  Train on the four problem patterns prior to training
on the other patterns to see if those four patterns can be learned.
If not, then it is probably either problematic input data (too much noise)
or a flawed learning algorithm. If it does learn the four patterns, then
look to see if it interferes with learning some of the other patterns
(asking for different responses to essentially the same pattern).

-George

