Evaluation

Finally, we are planning to carry out a larger user study. We must further verify that the differences in our study were not due to task difficulty differences since we did not control for the difficulty of finding a particular item. In particular, different values for the same constraints may not result in the same number of matching items. So even though two users have answered the same number of questions, the number of matching items for one user may be small enough for the system to begin presenting them, while the other user may need to answer additional questions first. To support an expanded evaluation, we have implemented a version of the system that recommends movies, which will let us draw from a broader user base. This should help us measure user satisfaction more easily, as Walker et al. (1998) have noted that efficiency is not the only important consideration, and that users might tend to prefer more predictable interfaces.