|(1) Two Goal|
|%Solved||100%||100%||100%||100% (6.0)||100% (6.0)||100% (6.0)|
|(2) Three Goal|
|% Solved||100%||100%||100%||100% (8.2)||100% (8.2)||100% (8.2)|
|(3) Four Goal|
|% Solved||100%||100%||100%||100% (10.3)||100% (10.3)||100% (10.3)|
These results are also summarized in Figure 10.
Table 2 records three different measures which reflect the effectiveness of replay. The first is the percentage of sequenced replay. Recall that replay of a trace is considered here to be sequenced if the skeletal plan is further refined to reach a solution to the new problem. The results point to the greater efficiency of replay in learning mode. In the domain, replay was entirely sequenced in this mode. In the transportation domain, retrieval based on failure did not always result in sequenced replay, but did so more often than in static mode.
The greater effectiveness of replay in learning mode is also indicated by the two other measures contained in the subsequent two rows of Table 2. These are respectively, the percentage of plan refinements on the final derivation path that were formed through guidance from replay (% Der), and the percentage of the total number of plans created through replay that remain in the final derivation path (% Rep). The case-based planner in learning mode showed as much or greater improvements according to these measures, demonstrating the relative effectiveness of guiding retrieval through a learning component based on replay failures. These results indicate that DERSNLP+EBL's integration of CBP and EBL is a promising approach when extra interacting goals hinder the success of replay.
In Section 4 we report on a more thorough evaluation of DERSNLP+EBL's learning component. This was conducted with the purpose of investigating if learning from case failure is of benefit for a planner solving random problems in a complex domain. For this evaluation we implemented the full case-based planning system along with novel case storage and adaptation strategies. In the next section, we describe the storage strategy that was developed for this evaluation.