NMRDPP is controlled by a command language, which is read either
from a file or interactively. The command language provides commands
for the different phases (preprocessing, expansion, solution) of the
methods, commands to inspect the resulting policy and value functions,
e.g. with rendering via DOT [1], as well as supporting
commands for timing and memory usage. A sample session, where the coin
NMRDP is successively solved with PLTLSTR and PLTLMIN is shown
in Figure 9.
NMRDPP is implemented in C++, and makes use of a number of
supporting libraries. In particular, it relies heavily on the CUDD
package for manipulating ADDs [43]: action specification
trees are converted into and stored as ADDs by the system, and
moreover the structured algorithms rely heavily on CUDD for ADD
computations. The state-based algorithms make use of the MTL - Matrix
Template Library for matrix operations. MTL takes advantage of modern
processor features such as MMX and SSE and provides efficient sparse
matrix operations. We believe that our implementations of MDP solution
methods are comparable with the state of the art. For instance, we
found that our implementation of SPUDD is comparable in performance
(within a factor of 2) to the reference implementation
[29]. On the other hand, we believe that data
structures used for regression and progression of temporal formulae
could be optimised.
Sylvie Thiebaux
2006-01-20