Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!udel!gatech!howland.reston.ans.net!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: BEGINNER:Forecasting/Time Series/Question
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <D2q0My.Avv@unx.sas.com>
Date: Fri, 20 Jan 1995 20:23:22 GMT
References:  <1732BCB4AS86.RVANRAAM@bcsc02.gov.bc.ca>
Nntp-Posting-Host: hotellng.unx.sas.com
Organization: SAS Institute Inc.
Lines: 94


In article <1732BCB4AS86.RVANRAAM@bcsc02.gov.bc.ca>, RVANRAAM@bcsc02.gov.bc.ca writes:
|> ...
|> Problem: Learn the y = sin(t) for t=t0,t1,...,tn using a neural
|> network.  After the network is trained, use it to predict
|> y = sin(t) into the future.
|>
|> Method:
|>
|> A) try feeding 5 past values of the sine function in to predict
|>    the 6th value using a 5x3x1 network.
...
|>  When I try to predict with it, it just gives me a few points which
|>  rise up and then a straight line after that.
...
|> |
|> |      * *
|> |   XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX  prediction
|> |  X        *                    *
|> | X          *                  *
|> |*            *                *
|> |--------------*-------------------------> t
|> |                *           *
|> |                  *       *
|> |                    *    *  actual
|> |                       *

You may need more training data. You might also try multiple starts from
random initial weights.

Two inputs and two hidden units are sufficient. Here's an example:

data sinem;
   do angle=0 to 55 by .1;
      if angle<10
         then sine=sin(angle)/2+.5;
         else sine=.;
      output;
   end;
run;

%netmodel(data=sinem,
   yvar=sine,                               /* target */
   xvar=lag5(sine) lag10(sine),             /* inputs */
   fitrange=angle to 10,                    /* select training data */
   hidden=2,                                /* # of hidden units */
   random=12, outfore=outfore);

Asterisks (*) are the training values; dashes (-) are the forecasts:

SINE |
 1.0 +  *       *       -      --      --      --      --      --      --
     | ***     *-*     --      --      --      --      --      --      --
     | * -     * *     - -     ---     ---     --      --     ---     - -
     | * *     * *     - -     - -     - -     - -    -  -    - -     - -
     | * *     * *     - -     - -    -  -    -  -    -  -    -  -    - --
     | * *    ** *    -  -    -  -    -  -    -  -       -    -  -    -  -
     |*   *   *  *-   -  -    -  -    -  -    -  -    -  -    -  -    -
     |*   *   *   *   -   -   -  -    -  -    -  -    -  -    -  -    -  -
     |*   *   *   *   -   -   -   -       -   -  -    -  -    -  -   --  -
 0.5 +*   *   *   *   -   -   -   -   -   -   -   -   -  -   -   -   -   -
     |    *   *   *   -   -   -   -   -   -  -       -    -  -    -  -   -
     |    *   *   *       -  --   -  -    -  -    -  -    -  -    -  -    -
     |    *  **   *  --   -  -    -  -    -  -    -  -    -       -  -    -
     |     * *    *- -    -  -    -  -       -    -  -    -  -       -    -
     |     * *     * -     - -    -- -    -  -    -  -    -  -    -  -    -
     |     * *     - -     - -     - -     - -    -  -    - -     - -     -
     |     * *     - -     - -     ---     ---     --      --     ---     -
     |     *-*     --      --      --      --      --      --      --      -
 0.0 +      *       -      --      --      --      --      --      --
     -+---------+---------+---------+---------+---------+---------+---------+-
      0         8        16        24        32        40        48        56

|>  How can the sigmoid
|>  functions give a cyclic curve anyway ?

You need to think in terms of the 5-dimensional embedding space.
The geometry is very different there.

|> I notice the NNDT network from ftp ftp.marcam.com
|> at /pub/pc/win3/demo/nndt100.zip does a good job of
|> learning the sin(t) function.  It also uses some BIAS at the
|> nodes which I don't have in my code.  Is a BIAS necessary ?

Bias is generally necessary. That would be the first thing to try.




-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
