Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: NN Vs Stats......
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <D31ADC.Mtn@unx.sas.com>
Date: Thu, 26 Jan 1995 22:27:12 GMT
References:  <srjg7930-2601950903020001@ruger-59.slip.uiuc.edu>
Nntp-Posting-Host: hotellng.unx.sas.com
Organization: SAS Institute Inc.
Lines: 85


In article <srjg7930-2601950903020001@ruger-59.slip.uiuc.edu>, srjg7930@uxa.cso.uiuc.edu writes:
|>
|> > In article <3g602q$hv5@vixen.cso.uiuc.edu>, srjg7930@uxa.cso.uiuc.edu
|> (johnson s) writes:
|> > |> Wait a minute.   Statistical analyses cannot recover a pattern which has
|> > |> missing parts or which is geometrically distorted (inverted, rotated, etc.)
|> > |> while neural networks can.
|> >
|> > Can you provide an example of a neural net that can do the above and is
|> > not a statistical model?
|>
|> You miss my point, I was asking if anyone could provide me with an example
|> of a statistical model which could recover a pattern with missing parts or
|> which is geometrically distorted.  I believe that there isn't any
|> statistical method to do so.

No, I didn't miss your point. I am a statistician, not an expert in
image processing or computer vision. I don't know what algorithms are
useful for these tasks, but if you gave me a description of an
algorithm, I could probably tell you whether it could be considered
statistical.

|> When one encounters missing data in any
|> typical regression analyses (just as an example) the usual procedures are
|> (short of tossing out the data or resampling) to fill in the missing data
|> by hand, i.e. by using the sample mean, assuming the missing data have the
|> same value as the last know datum point, etc.

There are various statistical methods for estimating ("imputing")
missing data. Many are implemented via the EM algorithm. Some require
distributional assumptions such as multivariate normality, others do
not.

|> ...
|> > |> Additionally, aren't most of the examples given in this thread all BP
|> > |> type networks?  I mean don't Kohonen and ART networks (and others) not
|> > |> use mean error gradiant type of learning?  And doesn't that exempt them
|> > |> from being classified as similar to statistical regression?
|> >
|> > Whether something uses a gradient descent form of learning is utterly
|> > irrelevant to whether it is a statistical method. While neural nets
|> > are often defined in terms of their algorithms or implementations,
|> > statistical methods are defined in terms of their results. The
|> > arithmetic mean, for example, can be computed by a (very simple)
|> > backprop net, by applying the usual formula SUM(x_i)/n, or by
|> > various other methods. What you get is still an arithmetic mean
|> > regardless of how you got there.
|> >
|>
|> No, strickly speaking statistical methods are *not* defined in terms of
|> their results.   The arithmetic mean is defined as:
|> SUM(x_i)/n, as you pointed out.  Statistical methods are mathematical
|> functions and so are defined by the arithmetic operations that compose
|> them.

You are confusing introductory statistical "cookbooks" with statistical
theory. Most statistics are defined as solutions to equations or
optimization problems. Even mathematical functions are not necessarily
defined by "arithmetic operations".

|> Most statistical methods take multi-dimensional data and reduce it
|> down to either a one dimensional expression, i.e. regression; or a zero
|> dimensional expression, i.e. correlation coefficient.  Neural network do
|> not reduce the dimensionality of the data (typically).

The usual feedforward nets do exactly the same thing as nonlinear
regression models, since they _are_ nonlinear regression models.

|> An ANN which can
|> take a photograph and recover missing or undistort distorted areas is
|> doing something that statistics cannot do and this is the advantage that
|> ANN's have.

As I said, I am not an expert in this area, but I find it hard to
believe that a neural net or any other AI technique could recover
missing areas on photographs unless you put extreme constraints on the
set of possible photographs and/or the size of the area. I would be
quite interested in any reliable information to the contrary.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
