Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!rochester!cornellcs!travelers.mail.cornell.edu!news.tc.cornell.edu!news.cac.psu.edu!news.pop.psu.edu!hudson.lm.com!newsfeed.pitt.edu!gatech!swrinde!cs.utexas.edu!usc!sdd.hp.com!hplabs!nsc!news
From: W Shields Neely <wneely@berlioz.nsc.com>
Subject: Re: COMMERICIALLY PRACTICAL NEURAL NETS? (Temporal Processing)
Message-ID: <D9G9Ls.MEI@nsc.nsc.com>
Sender: news@nsc.nsc.com (netnews maintenance)
Nntp-Posting-Host: neuron.nsc.com
Organization: National Semiconductor, Santa Clara
References: <D93KFC.CnI@nsc.nsc.com> <3q0rfd$l69@bigguy.eng.ufl.edu>
Date: Wed, 31 May 1995 15:56:15 GMT
Lines: 31

 
> The use of tapped delay lines (TDLs) has been successful in a number of temporal  
> applications, but will quickly lead to large networks with unrealistic training times. The  
> inherent problem is that each tap (i.e., time sample in the frame) requires its own weight.  
> The size of the frame (how much memory is needed) should be chosen based on the  
> dynamics in the input signal, whereas the number of weights within the network should be  
> based on the difficulty of the classification task. The answer is to use a memory structure  
> where the number of taps does not fix the memory depth (e.g., IIR rather than FIR). These  
> memory structures can still be applied as preprocessing stages, and will outperform the  
> TDL. Be aware, however, that the best solution is to adapt the memory depth based on  
> the data, which cannot be implemented as a preprocessor.
> 
> Curt Lefebvre
> NeuroDimension, Inc.
> curt@nd.com

Curt makes a number of good points in his posting. The use of IIR 
type configurations, where the output is fed back into an input, 
does offer many advantages, especially in trend line predictions. 
I have, however, found from practical cases that stability 
is a problem in continious mode signal processing with 
feedback configured networks. I would still use the TDL if I was 
new to the field, and trying to use Brainmaker to do some simple 
filtering or sequence detection. I would rather fight the network 
size and training time problems up front during development, than 
the stability problems, which may only show up in the field. I am 
in complete agreement with Curt that preprocessing with a static 
network can only do so much. 

W.Shields Neely
National Semiconductor
