Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!news.sprintlink.net!crash!mkppp.cts.com!user
From: Dean_Abbott@partech.com (dean abbott)
Subject: Re: Tree Based Algorithms
Organization: pgsc
Date: Wed, 1 Mar 1995 01:24:17 GMT
Message-ID: <Dean_Abbott-2802951731410001@mkppp.cts.com>
References: <3isr8d$q3i@lyra.csx.cam.ac.uk> <793975974snz@ecowar.demon.co.uk>
Sender: news@crash.cts.com (news subsystem)
Nntp-Posting-Host: mkppp.cts.com
Lines: 47

In article <793975974snz@ecowar.demon.co.uk>, jimmy@ecowar.demon.co.uk wrote:

> In article <3isr8d$q3i@lyra.csx.cam.ac.uk> srw1001@tulip.eng.cam.ac.uk writes:
> 
> ..
> >I am searching for references / ideas on tree growing / splitting
> >rules / pruning rules etc. with relation to tree based algorithms
> ..
>         I was told that AIM (Ivanenkho's GMDH) package by Abtech
>         employs a very powerful prunning algorithm based on research
>         by Barron et al. Unfortunately, that's all I know aboit it.
> 
>         Good luck
> 
>         Drago

AIM (AbTech Corp.), and it's parent ASPN (Barron Associates, Inc.) both use
the Predicted Squared Error (PSE) criterion for pruning.  PSE is the sum of the
fitting squared error (FSE) and a complexity penalty that is derived
statistically and is similar to the Akaike Information Criterion (AIC) and
the Minimum Description Length (MDL) criterion of Rissanen.  Basically, it
penalized for the number of degrees of freedom in the model, but the penalty
is smaller the more independent examples you have to determine you model.
In ASPN and AIM, PSE is used to prune the weights back in individual nodes, 
determine the number of layers in the network, and hence to determine when to
stop growing the network (it begins from simple single nodal element models).
The nodal elements used are polynomial (actually multinomial) based, including
higher order powers and cross-product terms.

But PSE (or MDL) can be used by any model synthesis method, not just polynomial
networks.  A couple of references are:

Barron, A.R., and R.L. Barron, "Statistical Learning Networks: A Unifying
View", 1988 Proceedings fo the 20th Symposium on the Interface.

Self-Organizing Methods in Modeling, S.J. Farlow (Ed.), Marcel Dekker, New
York, 
1984.

Hope this helps.

Dean

-- 
PAR Government Systems Corp.     |
1010 Prospect St., Suite 200     |
La Jolla, CA 92037               |
