Friday Apr 28
A Constructive Learning Network Based on Nonparametric Regression:
Receptive Field Locally Weighted Regression
Stefan Schaal, Georgia Tech
Methods from nonparametric statistics are the basis for one important
class of learning algorithms in which the data is modeled by means of
simple local functions. The local functions are either stored and a
predicted value is generated by a blending of neighboring local
functions, or the local functions are recalculated at the point where
a prediction is required from the data in memory. The former approach
is usually a computationally involved batch method and does not allow
incremental learning; however its lookup speed is fast. The latter
approach, in contrast, performs very fast incremental learning with
minimal interference, but it requires a computationally expensive
calculation at the moment a prediction is to be formed. Naturally the
question arises whether an incremental nonparametric learning system
can be accomplished which combines the best of both worlds.
Receptive Field Locally Weighted Regression (RFLWR) is an attempt to
achieve this goal. It represents the functional dependence of the
mapping from input to output data by a sparse distributed code which
is formed by a flexible number of local receptive fields. Each
receptive field consists of an activation function and a locally
linear output function. Parameters are adjusted incrementally by
second order methods. In this way, each receptive field finds its
own local distance metric, and it can cope with collinear data and
nuisance dimensions. There is no competition among the receptive
fields during learning and the size of the receptive fields reflects
the functional relation of the input-output mapping and not -- as in
several other receptive field methods -- merely the probability
distribution of the input data. These properties lead to robustness
towards dependent patterns of data sampling and avoid negative
interference to a large extent.
This talk will discuss the theoretical issues of RFLWR and demonstrate
the aforementioned properties by means of synthetic, visualizable data
sets.