OPTIMAL NN TOPOLOGY FOR TIME DEPENDENT PATTERNS
Nicolae Szirbik, Dept. of Computer Science
Technical University of Timisoara, Romania

                   Abstract

For a specific application, which is solved using the connectionist model, the
first step is to choose the most appropriate topology. Trying to apply NNs in
automated reasoning and robot command generation, we have proved some
advantages of recurrent topologies. Their principal quality is the temporal
dimension of functionality. Some difficulties with time dependent patterns
(e.g. the dilatation of the patterns) are implicitly solved by RNNs. However,
the gradient descent algorithms used to obtain a desired behavior of the RNN,
(according with the patterns from the training base) have a complexity of
O(pn^4), (p = number of patterns); under certain circumstances, it can be
reduced to O(pn^3). Even worse, the needed number of epochs for 
training cannot be known in advance.

A possible solution to overcome complexity is not to limit oneself to using
RNNs. For a _SPECIFIC_ application (e.g. a class of robotics applications),
we have used simple feedforward nets. There are some limitations and the
most important is that patterns cannot be infinitely cyclic. But for limited time
span patterns the approach is feasible. An artifice to "simulate" the temporal
dimension is necessary. An interesting conclusion is that performance can be
drastically improved by introducing a certain degree of recurrence. Applying
this kind of recurrence does not affect the O(pn^2) complexity of the training
algorithm. We tried also the unsupervised "winner-takes-all" strategy. The
conclusion has been that these nets can be used as an input "interface" for the
following feedforward net and with some original training rules we have
obtained excellent results (using a K-winners strategy). 

Our work in this area has been applied to the development of a pilot input
generator. The purpose of this application is to autonomously fly a plane into
high G-number maneuvers. The NN has to generate the trajectories imposed
for rudder, stick and throttle. One purpose of the talk is to discuss interesting
aspects concerning the training process of the command generator.The solving
manner of the pattern time dilatation and contraction will be presented.
Experimental results using different NN architectures will be discussed (RNN,
feedforward NN, "simple" recurrent NN, winner-takes-all NN).
---------------------------------------------------------------------