System Evaluation

As stated earlier, we believe that user modeling increases the effectiveness and efficiency of conversations with the system over time. To test this hypothesis, we carried out an experiment with a version of the ADAPTIVE PLACE ADVISOR that recommends restaurants in the San Francisco Bay Area. The system describes items using seven attributes: cuisine, rating, price, location, reservations, parking options, and payment options. Most attributes have few values, but cuisine and location have dozens. There are approximately 1900 items in the database. We asked several users, all from the Bay Area, to interact with the system to help them decide where to go out to eat. The users were given no external guidance or instructions on which types of restaurants to select, other than to look for and choose those that they might actually patronize. An experimenter was present during all these interactions, which were filmed, but his help was not needed except on rare occasions when a subject repeatedly tried words that were not included in the speech recognition grammar.