Infinite Models II: Mixtures of Experts and HMMs

Zoubin Ghahramani, Center for Automated Learning and Discovery, CMU


  I will briefly review the discussion of infinite models from my Feb 4 ML Lunch. I will then focus attention on two novel types of infinite models. Infinite Mixtures of Experts are a model that can be used for regression in situations where Gaussian Processes may be inadequate. Infinite-state HMMs are a nonparametric form of HMM in which the number of required hidden states, architecture, etc are determinined automatically. Both models make use of Markov chain techniques to sample over the latent variables while implicitly integrating out the (infinitely many) model parameters. Some small applications will be used to demonstrate these models. Finally, I will return to the question posed in the Feb 4 talk: should one use Occam's razor to find simple models, or should one ignore Occam's razor and focus on very large models?

Joint work with Carl E Rasmussen and Matthew J Beal.

Charles Rosenberg
Last modified: Wed Feb 6 09:23:47 EST 2002