Decision-Theoretic Planning with Asynchronous Events

Hakan L Younes


  I will introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, I will show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing MDP techniques. The techniques I will present can also be seen as an alternative approach for solving SMDPs, and I will demonstrate that the introduction of phases enables the generation of higher quality policies than those obtained by standard SMDP solution techniques. The talk is based on a paper that will be presented at AAAI-04. The paper is available on-line:

Back to the Main Page

Pradeep Ravikumar
Last modified: Thu Apr 8 08:22:07 EDT 2004