Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!oitnews.harvard.edu!purdue!lerc.nasa.gov!magnus.acs.ohio-state.edu!math.ohio-state.edu!howland.reston.ans.net!newsfeed.internetmci.com!news.sprintlink.net!redstone.interpath.net!sas!mozart.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: brittleness problem ...
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <DEnH1q.3qD@unx.sas.com>
Date: Sat, 9 Sep 1995 18:08:14 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <422eek$11r@cybernet.cse.fau.edu>
Organization: SAS Institute Inc.
Lines: 32


In article <422eek$11r@cybernet.cse.fau.edu>, uwechueo@cse.fau.edu (Okechukwu A. Uwechue) writes:
|> Does anyone know much about higher-order nets? I am using a third-order net for
|> my image processing research and I've come up against a problem: brittleness.
|> It seems that the net is not generalising for my test images, but it trains
|> correctly for the training set. 

It might be insufficient training data. It might be numerical
ill-conditioning. It might be a poor training algorithm. It might
be something else.

|> Each pixel in an image maps to a unique neuron
|> in the single-layer network so it is not possible to "prune" the net as would
|> be the case for optimizing the hidden layers in a MLP.

My understanding of "third-order network" is, in statistical
terminology, a linear model with main effects, two-way interactions,
and three-way interactions. If that is correct, you can prune it
unless you are using one of those elaborate constraint systems to
give translation invariance. But pruning tends not to work as well
as stopped training or Bayesian estimation.

|> Are multi-layer high-order nets a possible solution ?

That would just make things worse if any of my first three hypotheses
above are correct.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
