12:00, 18 June 1997, WeH 7220 A Large-Scale Test for Theory Refinement: The Wisconsin Adaptive Web Assistant Jude W. Shavlik University of Wisconsin - Madison (currently on sabbatical at U-MD and NLM) The last 5-10 years have seen activity in the machine learning subfield called theory refinement. Rather than solely learning from labeled input-output pairs, workers in this field develop algorithms that also make use of other knowledge about the task at hand (eg, approximately correct inference rules; a "domain theory"). While there have been a variety of promising laboratory demonstrations of the value of these algorithms (eg, higher accuracy with fewer training examples), these approaches have not yet had a practical impact nor been applied to any large problems. In order to investigate and improve the scaling properties of the theory-refinement perspective, we are constructing an instructable software agent that wanders the World-Wide Web and finds pages of interest. Advice -- domain-theory fragments -- occasionally provided by human users guide this software assistant, tailoring it to the individual interests of specific users. Reinforcements, such as retrieval times or users' ratings of retrieved pages, also mold the program. This testbed is also an example of more general perspective, namely instructable software that can be easily tailored to individual users and is also continually improved based on the feedback it receives during the course of its ordinary operation. (Note: this will be a work-in-progress talk about an implemented, prototype system that has not yet been thoroughly empirically evaluated.)