Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!godot.cc.duq.edu!news.duke.edu!news.mathworks.com!newsfeed.internetmci.com!news.sprintlink.net!new-news.sprintlink.net!news.interpath.net!sas!newshost.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: Thin plate spline function
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <DquxnA.7rx@unx.sas.com>
Date: Sat, 4 May 1996 01:33:10 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References: <DprC2z.20v@uns.bris.ac.uk> <Dq413s.DJu@aston.ac.uk>
Organization: SAS Institute Inc.
Lines: 86


M.J.Ratcliffe@bris.ac.uk wrote:
|> Several (and there are not many compared to the number of MLP/backprop 
|> refs...) RBF references blandly state that 'The Thin Plate Spline 
|> function does not require a spread constant...'. Why the bloody hell not? 

With an odd number of inputs, the spread constant would fall out of
the polynomial and be absorbed by the weights. Something similar must
happen for an even number of inputs, but exactly what escapes me at
the moment.

|> Also does anybody have any references on the differences between using 
|> different RBF functions - Gaussian, TPSF, multi-quadric, teapot, etc. 
|> 
|> Ideally these would be neural net-based RBF refs, rather than statistical 
|> theory.

Oh? And why would that be?

In article <Dq413s.DJu@aston.ac.uk>, mclachla@aston.ac.uk (Alan McLachlan) writes:
|> If you can get a copy, have a look at
|> 
|> "On the Use of Nonlocal and Non Positive Definite Basis Functions in
|> Radial Basis Function Networks"
|> 
|> David Lowe, 4th IEE International Conference on Artificial Neural Networks,
|> IEE publications no. 409, 1995, pp 206-211.

This article consists of arguments against various arguments against
using non-Gaussian RBFs. I don't see any good reasons in it _for_
using non-Gaussian RBFs.

|> A *brief* summary :
|> 
|> 1/ Non positive definiteness of the basis functions

This should be "positiveness", not "positive definiteness".

|> The first layer of an RBF attempts to model the unconditional distribution of
|> the data.

That depends on what kind of RBF net it is.

|> Any kernel-based density estimator constructed out of positive kernels and a 
|> finite set of samples cannot be unbiased, though the bias decreases 
|> asymptotically as the sample size increases.
|> 
|> Allowing kernels with negative regions improves the asymptotic convergence
|> and potentially gives smaller bias (zero bias if you're lucky).

True in theory, but in practice, there seems to be little benefit to
higher-order kernels. And this argument is relevant only to a restricted
class of non-positive RBFs.

|> 2/ Non locality
|> 
|> Even with unbounded basis functions, the RBF network can give a localised
|> response (if you view things in terms of an appropriate basis). 

But that view of things seems to be of little use. In terms of the weights
in the network, the response is not localized.

|> In the 
|> particular case where you attempt to model a polynomial using an RBF with a 
|> regular grid of centres, you *must* have unbounded basis functions.

That's only for exact interpolation.

|> From the point of view of both bias and universality, unbounded, non-positive
|> definite basis functions have the potential to outperform everyone's 
|> favourite (bounded, positive-definite) Gaussian. The thin-plate spine 
|> (r*r*logr) is a good candidate on both counts.

Thin-plate spines are good, but note that that formula (r^2*log r^2) applies
only with 2 inputs.

|> As for the width parameter, this is a hyperparameter which is not essential
|> for the purposes of universality in *any* radial basis function, 

But it is essential for all practical applications.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
