It is important to note that when we refer to macro-actions generated and used by Marvin these are all generated during the planning process for that specific problem. No additional learning time or knowledge gained from solving other problems was used by Marvin in the competition, or in producing the additional results presented in this paper. Although some planners can use additional `learning time' when solving a series of problems, a satisfactory way to incorporate this extra time into the time taken to solve each problem, as measured in the planning competition, has yet to be found. In the planning competition the planners are compared based on their performance on isolated problem instances, which is still an interesting comparison to make.
The results presented were produced on two machines: a machine at the University of Strathclyde (with a 3.4GHz Pentium 4 processor) and the IPC 4 competition machine (with a 3GHz Xeon processor). In both cases, the planner was subjected to a 30 minute time limit and a 1Gb memory usage limit. All results that are directly compared against each other (i.e. appear on the same graph) are produced on the same machine. The domains used for evaluation are taken from IPC 3 and IPC 4, and are described in detail in the papers giving an overview of each of the two competitions [Long & Fox 2003, Hoffmann & Edelkamp 2005].