Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!gatech!howland.reston.ans.net!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: bias
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <D7t3E1.ECz@unx.sas.com>
Date: Sat, 29 Apr 1995 17:03:37 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <1995Apr28.195623.7533@cm.cf.ac.uk>
Organization: SAS Institute Inc.
Lines: 34


In article <1995Apr28.195623.7533@cm.cf.ac.uk>, C.M.Sully@cm.cf.ac.uk (Chris Sully) writes:
|> ...
|> I've trained a backpropagation ANN to predict the birthweight of babies given
|> 11 predictor variables. ...
|>
|> The statisticians who provided the data preferred to see the results in terms
|> of residuals (actual outputs-predicted outputs). They expected that the
|> mean of the residuals would be zero. It wasn't, being around 10-20 grammes
|> (typical birthweights being around 4000 grammes if I recall correctly).

If the network is trained to convergence by least squares and there is a
"bias" (in the usual NN sense, "intercept" in statisticalese) in the
outputs, then the residuals will indeed have a mean of zero. If you
violate any of those conditions, the residuals may or may not have a
mean of zero. The statisticians should realize that, if they understand
that feedforward neural nets are nonlinear regression models.

|> It seems there is a degree of bias in the model, with the model consisting
|> of two parts, the data and the neural network.

The term "bias" here is in the statistical sense. Various methods of
training neural nets do produce biased estimates. For example, weight
decay and training with noise/jitter added to the inputs are two
extensions of ridge regression to nonlinear models (see my post from
yesterday on Re: How do YOU apply noise for training?), and ridge
regression is one of the most common forms of biased estimation used
by statisticians.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
