Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news.harvard.edu!news2.near.net!MathWorks.Com!europa.eng.gtefsd.com!howland.reston.ans.net!EU.net!relay.puug.pt!unl.fct.unl.pt!tr
From: tr@fct.unl.pt (Thomas Rauber)
Subject: Re: Stopped Training
Message-ID: <1994Sep16.131944.2099@fct.unl.pt>
Sender: news@fct.unl.pt (USENET News System)
Organization: F.C.T. - Universidade Nova de Lisboa
X-Newsreader: TIN [version 1.2 PL0]
References: <Cw6xJA.1Gr@unx.sas.com>
Date: Fri, 16 Sep 1994 13:19:44 GMT
Lines: 36

Warren Sarle (saswss@hotellng.unx.sas.com) wrote:

: There are many methods for estimating generalization error:
:  * AIC, SBC, FPE, Mallows' C_p, etc.--fast but require "large" sample
:  * Split-sample validation--fast but statistically inefficient
:  * Cross-validation (leave one out)--slow and erratic
                                                ^^^^^^

Would you please specify in more detail this attribute of the leave-one-out
estimate. I have used the method in supervised classification because I assumed that
is gives quite a good idea about the estimated error.

From
  Devijver, P. A., and Kittler, J., "Pattern Recognition --- A Statistical
  Approach," Prentice/Hall Int., London, 1982.
page 356:

.. The leave-one-out error estimate has been found experimentally to be
approximated unbiased, whatever be the classifier and the underlying
distributions. ...
.. In counterpart the leave-one-out method suffers from at least two
disadvantages ... increase in the variance of the estimator ...excessive
computation. ...



--
 ________________________________________________________________________
|                                  |                                     |
|         Thomas W. Rauber         | E-mail:  tr@fct.unl.pt              |
|__________________________________|                                     |
|   Universidade Nova de Lisboa    | Fax:   (+351) (1) 295-7786          |
|         & UNINOVA  -             |                                     |
|  Center for Intelligent Robotics | Phone: (+351) (1) 294-1085          |
|  2825 Monte Caparica, PORTUGAL   |                   Ext. 2228         |
|__________________________________|_____________________________________|
