Newsgroups: sci.math.num-analysis,comp.ai.neural-nets,sci.stat.math
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!gatech!news.mathworks.com!uunet!in2.uu.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: Questions about my new NN program
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <Dnrpov.1rK@unx.sas.com>
Date: Tue, 5 Mar 1996 00:07:43 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References: <4gr0p7$iqc@news2.deltanet.com> <21c7cc$11223.178@www.qed.com> <4h3n2h$lid@cloner2.ix.netcom.com> <313866b0.769609@news>
Organization: SAS Institute Inc.
Followup-To: comp.ai.neural-nets 
Lines: 143
Xref: glinda.oz.cs.cmu.edu sci.math.num-analysis:26947 comp.ai.neural-nets:30285 sci.stat.math:9556


In article <313866b0.769609@news>, steve@tropheus.demon.co.uk (Stephen Wolstenholme) writes:
|> On 29 Feb 1996 08:10:57 GMT, jdadson@ix.netcom.com(Jive Dadson )
|> wrote:
|> 
|> >An NN with 4 hidden units
|> >produces a beautiful cycle of a sine wave when trained on
|> >only 10 (noiseless) data points. (Oddly, increasing the
|> >number of data-points to 50 does not seem to improve the
|> >fit to the true sine wave. I guess the 4 hidden units are
|> >maxed out on what they can do.)
|> >
|> You'll find that if you use even less hidden units that the number of
|> data points required increases. If you use 3 then you will need about
|> 30 data points to get a nice clean cycle.

What? The fewer hidden units, the more data you need? Isn't that
backwards?

Here are results from 11 training cases from networks with 4, 3, and 2
hidden units. The Os represent network outputs, and the -s are test cases
that don't coincide with the outputs to the resolution of the plain
ascii plot. If there were any training cases that didn't coincide
with Os, they would be shown as 1s, but there aren't any.

Note Followup-To comp.ai.neural-nets only.

                                4 Hidden Units

    1.0 +                                           OOOOO
        |                                         OO    OO
        |                                        OO       O
        |                                       OO         O
        |                                      OO           O
        |                                      O             O
        |                                     O              O-
    0.5 +                                    OO               O
        |                                    O                 O
        |                                   O                  O
    _P1 |                                   O                   O
        |                                  O                    OO
        |                                 O                      O
        |                                 O                       O
    0.0 +       O                        O                        O
        |        O                       O
        |        O                      O
        |         O                    OO
        |         OO                   O
        |          O                  O
        |           O                 O
   -0.5 +           O                O
        |            O              OO
        |             O             O
        |             OO           O
        |              OO         O
        |               OO       O
        |                OO    OO
   -1.0 +                  OOOOO
        ---+---------+---------+---------+---------+---------+---------+--
        -3.768    -2.512    -1.256     0.000     1.256     2.512     3.768

                                       ANGLE


                                3 Hidden Units

    1.0 +                                           OOOOO
        |                                         OO    OO
        |                                        OO       O
        |                                       OO         O
        |                                      OO           O
        |                                      O             O
        |                                     O              OO
    0.5 +                                    OO               O
        |                                    O                 O
        |                                   O                  O
    _P1 |                                   O                   O
        |                                  O                     O
        |                                 O                      O
        |                                 O                       O
    0.0 +       O                        O                        O
        |        O                       O
        |        O                      O
        |         O                    OO
        |         OO                   O
        |          O                  O
        |           O                 O
   -0.5 +           O                O
        |            O              O
        |             O             O
        |             OO           O
        |              OO         O
        |               OO       O
        |                OO    OO
   -1.0 +                  OOOOO
        ---+---------+---------+---------+---------+---------+---------+--
        -3.768    -2.512    -1.256     0.000     1.256     2.512     3.768

                                       ANGLE



                                2 Hidden Units

    1.0 +                                           OOOO-
        |                                         OO    OO
        |                                        OO       O
        |                                       OO         O
        |                                      OO           O
        |                                      O             O
        |                                     O              OO
    0.5 +                                    O-               O
        |                                    O                 O
        |                                   O                  O
    _P1 |                                  OO                   O
        |                                  O                     O
        |                                 O                      O
        |                                 O                       O
    0.0 +       O                        O                        O
        |        O                       O
        |        O                      O
        |         O                    -O
        |         OO                   O
        |          O                  O
        |           O                 O
   -0.5 +           O                O
        |            O              OO
        |             O             O
        |             OO           O
        |              OO         O
        |               OO       O
        |                OOO   OO
   -1.0 +                  OOOOO
        ---+---------+---------+---------+---------+---------+---------+--
        -3.768    -2.512    -1.256     0.000     1.256     2.512     3.768

                                       ANGLE

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
