Our first endeavor was to find an appropriate memory size for our task. Memory size corresponds to degree of generalization of training data. We ran experiments with the defender moving at two different speeds and with the agent having various memory sizes. Table 1 presents the results.
Table 1: This table shows success rate during 1000 test trials after
1000 trials of training as well as the number of training trials it
took to reach a success rate of 70% overall (after at least 100
trials). Trials that never reached the 70% mark are marked as
``-''. The ideal memory size would exhibit a speed-independent high
success rate and quick learning rate (low number of trials to reach
70% success). The smaller memories tend to learn quicker
but not as well, while the larger memories learn better but more slowly.
These numbers show the tradeoff between better performance in the long run (larger memory), and quicker learning (smaller memory). Smaller memories enable quicker learning due to the fact that the retrieval slots are fixed. Notice that there is no memory size that is absolutely better than all others, but there are certainly some sizes that are worse. From these trials we determined that, for this particular task, a memory of size 18 is dense enough to give good performance and sparse enough to learn relatively quickly. For the remainder of our experiments, we ran trials both with memories of size 18 and memories of size 360 in order to further compare the results. Repeatedly, we found that the smaller, more generalized memory gave faster learning without sacrificing much long-term performance.