Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!oitnews.harvard.edu!newsfeed.rice.edu!news.sesqui.net!cpk-news-hub1.bbnplanet.com!news.bbnplanet.com!cam-news-hub1.bbnplanet.com!howland.erols.net!torn!kwon!watserv3.uwaterloo.ca!news
From: Rasmus Jensen <RTBJENSE@MANSCI.Watstar.UWaterloo.CA>
Subject: Re: Beginners questions
Content-Type: text/plain; charset=us-ascii
Message-ID: <329BAD56.5D2@MANSCI.Watstar.UWaterloo.CA>
Sender: news@watserv3.uwaterloo.ca
Nntp-Posting-Host: graybore.watstar.uwaterloo.ca
Content-Transfer-Encoding: 7bit
Organization: University of Waterloo
References: <32998DF2.41C6@dxcoms.cern.ch> <57d1eo$pik@crc-news.doc.ca> <329AE2A8.41C6@dxcoms.cern.ch> <57euvb$fgt@crc-news.doc.ca>
Mime-Version: 1.0
Date: Wed, 27 Nov 1996 02:54:14 GMT
X-Mailer: Mozilla 3.0 (Win16; I)
Lines: 61

Hi everyone,

I'm not relly into NN yet, but hopefully I will be next term when I take
a NN-course.

Anyway, when I read this I got an idea, see below.

Stefan C. Kremer wrote:
> In article <329AE2A8.41C6@dxcoms.cern.ch>, gamble@dxcoms.cern.ch says...
> >There again, by looking at the local surface gradient as I hop I can
> >build up some confidence concerning the local terrain. (I don't see
> >how to do this easily).
> >If the optimum is really an isolated steep-sided hole in a hill, then
> >it is going to be difficult for any step-wise approximating algorithm
> >to find it.
> 
> Right.

Snip!

> >If we are looking for a minimum, and this happens to be the crater of
> >a volcano, then I don't see how any gradient based algorithm can
> >find it - without an exhaustive search - or good luck!.
> 
> Yes, exactly!  

Do you really have to do an exhaustive search if a minimum (local or
global, I don't know) is in a up-hill crater. What about the following
(which will be in very non-technical language):

1. Have a frog/kangaroo that you drop into weight-space. The frog
follows the gradient down-hill to a minimum, which then probably is a
local minimum.
(Now comes the funny part!)
2. Make the frog breeth one "goat" (or whatever I shall call them) for
each possible direction to go. For example, in a two-dimensional
weight-space it would be 4 goats which would go north, east, west and
south. Set the goats of by a little distance from the frog so they find
a gradient and can start to climb. Eventually they will all reach
maximums.
3. This step is the reverse of step 2. Every goat breeths new frogs that
will start to find the surrounding valleys.
4. Goto step 2.

To prevent overpopulating the weight space, if an animal ends up in a
minimum or maximum already populated by a frog or goat, it will
annihilate (I feel I must be more scientific by adding a bit of atom
physics to this nice fable).

When all this have gone on for N cycles, all the minimums (and maximums
as well) should be populated, and the frog in the deepest valley wins!

Comments are wellcome.
-- 
Rasmus Jensen

Dept. of Management Sciences                    Carl Pollock Hall 4316
University of Waterloo                          Fax: +1-(519)-746-7252
Waterloo, Ont.                          Phone, home: +1-(519)-725-6060
N2L 3G1                     Phone, office: +1-(519)-885-1211 ext. 3863
Canada                     Email: rtbjense@mansci.watstar.uwaterloo.ca
