next up previous
Next: Planners Up: Problem Assumptions Previous: Problem Assumption 2: How

Problem Assumption 3: Does Performance Depend on PDDL Requirements Features?

The planners were all intended to handle STRIPS problems. Some of the problems in the test set claim to require features other than STRIPS; one would expect that some of the planners would not be able to handle those problems. In addition, those planners that claim to be able to handle a given feature may not do as well as other planners. Table 6 shows the effects of feature requirements on the ability to solve problems. The data in this table are based on the features specified with the :requirements list in the PDDL definition of the domain.


Table 6: The number of problems claiming to require each PDDL feature solved by each planner.
Planner Feature
  Axioms Cond. Eff. Dis. Pre. Equality Exists Pre. Safety Strips Typing Forall Pre.
A 0 0 0 35 0 0 255 8 0
B 0 0 0 8 0 0 268 0 0
C 5 169 165 216 163 2 561 197 160
D 3 164 166 196 139 0 279 180 139
E 1 162 152 199 157 0 384 168 149
F 0 157 145 185 150 0 376 165 145
G 0 0 0 8 0 0 276 0 0
H 0 0 0 46 0 0 285 17 0
I 0 138 138 169 138 0 441 139 138
J 0 130 130 160 130 0 502 130 130
K 0 0 0 8 0 0 240 0 0
L 0 19 13 24 16 0 421 13 13
M 0 168 169 212 149 2 372 180 151


We did not verify that the requirements were accurate or necessary; thus, the problem may be solvable by ignoring a part of the PDDL syntax that is not understood, or the problem may have been mislabeled by its designer. This is evident in cases where a planner that does not support a given feature still appears to be able to solve the corresponding problem. Some planners, e.g., older versions of STAN, will reject any problem that requires more than STRIPS without trying to solve it; an ADL problem that only makes use of STRIPS features would not be attempted.

As guidance on which planner to use when, these results must be viewed with some skepticism. For example, it would appear based on these results that planner I might be a good choice for problems with conditional effects as it was able to solve many of these problems. This would be a mistake, since that planner cannot actually handle these types of problems. In these cases, the problems claim to require ADL, but in fact, they only make use of the STRIPS subset.

Clearly, certain problems can only be solved by specific planners. For instance, C and M are the only planners that are able to handle safety constraints, while based on the data, only C, D and E appear to handle domain axioms. About half the planners had trouble with the typed problems. Some of the gaps appear to be due to problems in the translation to native representation.


next up previous
Next: Planners Up: Problem Assumptions Previous: Problem Assumption 2: How
©2002 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.