next up previous
Next: Controlled Testing Up: Results Previous: Results

The Competition

ATTac-2000's scores in the 88 preliminary-round games ranged from -3000to over 4500 (mean 2700, std. dev. 1600). A good score in a game instance is in the 3000 to 4000 range. We noticed that there were many very bad scores (12 less than 1000 and seven less than 0). This is largely the result of ATTac-2000 not yet being imbued with its adaptive timing of bidding modes. During the preliminary round, ATTac-2000 shifted from passive to active bidding mode with 50 seconds left in the game instance. While 50 seconds is usually plenty of time to allow for at least 2 iterations through ATTac-2000's bidding loop, there were occasions in which the network and server lags were such that it would take more than 50 seconds to obtain updated market prices and submit bids. In this case, ATTac-2000 would either fail to buy airline tickets, or worse still, would buy airline tickets but not get the final hotel bids in on time. Noticing that the server lag tended to be consistent within a game instance (perhaps due to the traffic patterns generated by the participating agents), we introduced the adaptive timing of bidding modes described in Section 3.3. After this change, ATTac-2000 was always able to complete at least one, and usually two, bidding loops in the active bidding phase.

The adaptive allocation strategy never came into play in the finals, as ATTac-2000 was able to optimally solve all of the allocation problems that came up during the finals very quickly using the integer linear programming method.

However, the adaptive hotel bidding did play a big role. ATTac-2000 performed as well as the other best teams in the early TAC games when hotel prices (surprisingly) stayed low, and then out-performed the competitors in the final games of the tournament when hotel prices suddenly rose to high levels. Indeed, in the last 2 games, some of the popular hotels closed at over $400. ATTac-2000 steered clear of these hotel rooms more effectively than its closest competitors.

Table 4 shows the scores of the 8 TAC finalists [Wellman, Wurman, O'Malley, Bangera, Lin, Reeves, WalshWellman et al.2001]. ATTac-2000's consistency (std. dev. 443 as opposed to 1600 in the preliminaries) is apparent: it avoided having any disastrous games, presumably due in large part to its adaptivity regarding timing and hotel bidding.

Table 4: The scores of the 8 TAC finalists in the semi-finals and finals (13 games).
Rank Team Avg. Score Std. Dev. Institution
1 ATTac-2000 3398 443 AT&T Labs - Research
2 RoxyBot 3283 545 Brown University, NASA Ames Research
3 aster 3068 493 STAR Lab, InterTrust Technologies
4 umbctac1 3051 1123 University of Maryland at Baltimore County
5 ALTA 2198 1328 Artificial Life, Inc.
6 m_rajatish 1873 1657 University of Tulsa
7 RiskPro 1570 1607 Royal Inst. Technology, Stockholm University
8 T1 1167 1593 Swedish Inst. Computer Science, Industilogik

next up previous
Next: Controlled Testing Up: Results Previous: Results
Peter Stone