Newsgroups: comp.speech
Path: pavo.csi.cam.ac.uk!pipex!uunet!charon.amdahl.com!pacbell.com!iggy.GW.Vitalink.COM!cs.widener.edu!eff!sol.ctr.columbia.edu!ira.uka.de!fauern!fauna!immd5.informatik.uni-erlangen.de!wieckert
From: wieckert@immd5.informatik.uni-erlangen.de (Wieland Eckert)
Subject: Re: HMM with null transitions
References: <1992Oct5.192354.9578@seas.gwu.edu> <NILES.92Oct20085734@pizzicato.parc.xerox.com>
Message-ID: <BwHFGq.9C3@immd4.informatik.uni-erlangen.de>
Sender: news@immd4.informatik.uni-erlangen.de
Organization: CSD., University of Erlangen, Germany
Date: Wed, 21 Oct 1992 17:34:01 GMT
Lines: 67

niles@parc.xerox.com (Les Niles) writes:

>In article <1992Oct5.192354.9578@seas.gwu.edu> marshall@seas.gwu.edu (Christopher Marshall) writes:
>   I have been waiting for just such a newsgroup.  I am just starting
>   my master's thesis in speech recognition and I have been studying
>   ...
>   I am trying to figure out how to solve the evaluation problem
>   for an HMM with null transitions.

>Null transitions (or the equivalent null states in models that associate
>symbols with states) are useful for gluing things together, and they
>can save computation.

Indeed they can save a lot of either computation time or space
or both. In my master's thesis I've designed and implemented
null transitions ('pseudo states') for a great reduction of
space requirement when constructing higher models of HMMS.

Have:
1. the concatenation of HMMs
	H1 := Ha Hb
2. the alternative of HMMs
	H2 := Ha | Hb
3. the loop over a HMM
	H3 := Ha+

and see what happens!
Case 2 is fairly trivial and gives you no major headaches.
Case 1 says, one has to connect every final state of Ha to each
initial state of Hb (with given probabilities and emmission
densities) which produces a*b edges.
Case 3 is nearly the same, just connect all final nodes with
all initial nodes of the same HMM. This makes ~ a*a edges.

When constructing models generative-grammar-like, you get easily
really big amounts of edges:
	CITY := bremen | hamburg | frankfurt | ...
	SENTENCE := i want to go to CITY
	i := /i/        (basic HMM)
When modeling a unit (say hamburg), of course you have to
consider several alternative models 
	hamburg := ( /ham/ /burk/) | ( /ham/ /bur/ /k/) | ...
according to best composition / training results.

Now imagine the number of edges growing.   :-))

Solution:
- insert a 'pseudo state' which just collects the OR-results of
the combination of alternatives
- when searching in this 'pseudo HMM' think of the pseudo states
living in the space between ]t,t+1[  (excluded)
- do a time synchronous search (I did beam search)
- don't forget them when the last time slice is reached
- then incidentially you get the right value in the top node
(here SENTENCE) for FREE, no longer search for an ending node

We got a really big saving in process space because of the
complex structure of the higher levels of models. When run
on (nearly) linear models you loose a bit of processing time.
When run on complex and big models you can get lots of reduction.

Feel free to ask me further details.
W/

-----
Wieland Eckert, Universitaet Erlangen-Nuernberg, Mustererkennung
e-mail: wieland.eckert@informatik.uni-erlangen.de
