Newsgroups: comp.speech
Path: lyra.csx.cam.ac.uk!warwick!pipex!howland.reston.ans.net!vixen.cso.uiuc.edu!uwm.edu!daffy!uwvax!sinetnews!news.u-tokyo.ac.jp!news.tisn.ad.jp!s.u-tokyo!kuis-news!wnoc-kyo-news!atrwide!atr-la!lucke
From: lucke@itl.atr.co.jp (Helmut Lucke)
Subject: Re: A stupid question on predictor order
In-Reply-To: ajr@eng.cam.ac.uk's message of 09 Apr 1994 17:17:15 GMT
Message-ID: <LUCKE.94Apr13140519@atrq28.itl.atr.co.jp>
Sender: news@itl.atr.co.jp (USENET News System)
Nntp-Posting-Host: atrq28
Organization: ATR Telecommunications Research Labs., Kyoto, JAPAN
References: <Cnz085.B6r@ecf.toronto.edu> <AJR.94Apr9181715@dsl.eng.cam.ac.uk>
Date: Wed, 13 Apr 1994 05:05:19 GMT
Lines: 37


Tony Robinson writes:

   In article <Cnz085.B6r@ecf.toronto.edu> liuca@ecf.toronto.edu (LIU  CHEUNG WAH STEPHEN) writes:
   >
   > 	Generally speaking, is it true that the higher the order the predictor
   > has, the more accurate the prediction will be? If true, why?

   Assuming we are speaking of standard linear prediction here, then the
   answer is yes.  The expected squared prediction error will decrease (or
   stay the same) with increasing number of parameters.

   Why?  Well, if you have an algorithm that gives you the global optimum
   for an optimisation problem, and adding parameters with value zero does
   not change the solution, then allowing these parameters to be non-zero
   can only help.


This is true only if there is enough data to estimate all predictor
coefficients. I beleive that in statistical economics, where linear
predictive coding is also used, people do statistical tests to verify each
additional coefficient. That is to say they check whether the inclusion of
another coefficient is likely to reduce the expected error for unseen data.
Naturally, for the training data the error always reduces with the 
inclusion of a new coefficient. 

For some reason in speech recogniton such tests are not performed -- 
rather the designer chooses a fixed number which he believes is right. 


--------------------------------------------------------------------
Helmut Lucke                                <lucke@itl.atr.co.jp>
ATR Interpreting Telecommunications Research Laboratories
2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-02, JAPAN
Tel: +81-7749-5-1382 (direct)               Fax:   +81-7749-5-1308
     +81-7749-5-1301 (switchboard)
--------------------------------------------------------------------
