Newsgroups: comp.speech
Path: lyra.csx.cam.ac.uk!doc.ic.ac.uk!aixssc.uk.ibm.com!ibm.de!Munich.Germany.EU.net!Germany.EU.net!EU.net!howland.reston.ans.net!cs.utexas.edu!convex!seas.smu.edu!vivaldi!netkeeper.sj.nec.com!nec-gw!nec-tyo!wnoc-tyo-news!aist-nara!wnoc-kyo-news!atrwide!atr-la!lucke
From: lucke@itl.atr.co.jp (Helmut Lucke)
Subject: Re: HMM-isolated word recognizer
In-Reply-To: opt1@rz.tu-ilmenau.de's message of 4 Jul 1994 21:59:10 GMT
Message-ID: <LUCKE.94Jul8105817@atrq28.itl.atr.co.jp>
Sender: news@itl.atr.co.jp (USENET News System)
Nntp-Posting-Host: atrq28
Organization: ATR Telecommunications Research Labs., Kyoto, JAPAN
References: <2va0ne$hhj@prakinf2.PrakInf.TU-Ilmenau.DE>
Date: Fri, 8 Jul 1994 01:58:17 GMT
Lines: 35


opt1@rz.tu-ilmenau.de (Praktikum DOT) writes:

>  2. At some points of iteration the procedure seems to provide a negative difference (log P' - log P) (where P' is the probability after the iteration step). I think this is possible because one computes the log P score by the Viterbi procedure and so gets only a part of the whole P (i.e. log P(observation,optimal sequence)). BUT where shall I stop the iteration? At the same step where the gain got negative or later on when the gain became positive again or should I check just the absolute value of th

>  ?

Whether you use Viterbi decoding or forward-backward calculations, your
likelihood should always increase. If it doesn't you certainly have
a bug in your programme. My advice: check your code again.


The reason why Likelihood monotonically increases for the Viterbi training
as well, is simple:

Consider first the problem of supervised training where we are given
the `correct' state sequence. In this case the re-estimation 
equations will give you a monotonically increasing likelihood for
each iteration. (Infact since the HMM is no longer `hidden' you will
get the optimal parameters in the first iteration and the parameters will
remain constant from then onwards, but this is not the point here).

Now if at each iteration you are allowed to choose the optimal
path you can only do better: The same state path as the previous
iteration would already result in a likelihood at least as good as in the
previous iteration. If the viterbi algorithm yealds an even better path, then
so be it. The Likelihood  must be even higher in this case.

--------------------------------------------------------------------
Helmut Lucke                                <lucke@itl.atr.co.jp>
ATR Interpreting Telecommunications Research Laboratories
2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-02, JAPAN
Tel: +81-7749-5-1382 (direct)               Fax:   +81-7749-5-1308
     +81-7749-5-1301 (switchboard)
--------------------------------------------------------------------
