Workshop on "Intelligent Tutoring Systems for Ill-Defined Domains"



Intelligent Tutoring Systems have made great strides in recent years. Robust ITSs have been developed and deployed in arenas ranging from mathematics and physics to engineering and chemistry. Over the past decade intelligent tutoring systems have become increasingly accepted as viable teaching and learning tools in academia and industry. 

Most of the ITS research and development to this point has been done in well-defined domains. Well-defined domains are characterized by a basic formal theory or clear-cut domain model. Such domains are typically quantitative, and are often taught by human tutors using problems where answers can unambiguously be classified as correct or incorrect. Well-defined domains are particularly amenable to model-tracing tutoring systems. Operationalizing the domain theory makes it possible to identify study problems, provide a clear problem solving strategy, and assess results definitively based on the existence of unambiguous answers. Help can be readily provided by comparing the students’ problem-solving steps to the existing domain models. 

Not all domains of teaching and inquiry are well-defined, indeed most are not. Domains such as law, argumentation, history, art, medicine, and design are ill-defined. Often even well-defined domains are increasingly ill-defined at the edges where new knowledge is being discovered. Ill-defined domains lack well-defined models and formal theories that can be operationalized, typically problems do not have clear and unambiguous solutions. For this reason ill-defined domains are typically taught by human tutors using exploratory, collaborative, or Socratic instruction techniques.

Ill-defined domains present a number of unique challenges for researchers in Intelligent Tutoring Systems and Computer Modeling. These challenges include 1) Defining a viable computational model for aspects of underspecified or open-ended domains; 2) Development of feasible strategies for search and inference in such domains; 3) Provision of feedback when the problem-solving model is not definitive; 4) Structuring of learning experiences in the absence of a clear problem, strategy, and answer; 5) User models that accommodate the uncertainty of ill-defined domains; and 6) User interface design for ITSs in ill-defined domains where usually the learner needs to be creative in his actions, but the system still has to be able to analyze them.

These challenges must be faced if the ITS community is ever to branch out from the traditional domains into newer arenas. Over the past few years a number of researchers have begun work in ill-defined domains including law, medicine, professional ethics and design. This workshop represents a chance to share what has been learned by those practitioners at a time when work in these domains is still nascent. 


We invite work at all stages of development, including particularly innovative approaches in their early phases. Research papers (up to 9 pages) and demonstrations (up to 4 pages, describing an application or other work to be demonstrated live at the workshop) are welcome for submission. Workshop topics include but are not limited to:
  • Model Development: Production of formal or informal models of ill-defined domains or subsets of such domains.
  • Teaching Strategies: Development of teaching strategies for such domains, for example, Socratic, problem-based, task-based, or exploratory strategies.
  • Search and Inference Strategies: Identification of exploration and inference strategies for ill-defined domains such as heuristic searches and case-based comparisons.
  • Assessment: Development of Student and Tutor assessment strategies for ill-defined domains. These may include, for example, studies of related-problem transfer and qualitative assessments.
  • Feedback: Identification of feedback and guidance strategies for ill-defined domains. These may include, for example, Socratic (question-based) methods or related-problem transfer. 
  • Exploratory Systems: Development of intelligent tutoring systems for open-ended domains. These may include, for example, user-driven “exploration models” and constructivist approaches.
  • Collaboration: The use of peer-collaboration within ill-defined domains, e.g., to ameliorate modeling issues.
  • Representation: Free form text is often the most appropriate representation for problems and answers in ill-defined domains; intelligent tutoring systems need techniques for accommodating that. 
The topics can be approached from different perspectives: theoretical, systems engineering, application oriented, case study, system evaluation, etc.


Vincent Aleven, Carnegie Mellon University, USA
Jerry Andriessen, University of Utrecht, The Netherlands
Kevin Ashley, University of Pittsburgh, USA
Michael Baker, Centre National de la Recherche Scientifique, France
Paul Brna, University of Glasgow, UK
Robin Burke, DePaul University, USA
Jill Burstein, Educational Testing Service, USA
Rebecca Crowley, University of Pittsburgh, USA
Susanne Lajoie, McGill University, Canada
Collin Lynch, University of Pittsburgh, USA
Liz Masterman, Oxford University, UK
Bruce McLaren,
Carnegie Mellon University, USA
Antoinette Muntjewerff, University of Amsterdam, The Netherlands
Katsumi Nitta, Tokyo Institute of Technology, Japan
Niels Pinkwart, Carnegie Mellon University, USA
Beverly Woolf, University of Massachusetts, USA


8.30 - 8.40 Introduction

8.40 - 9.45 Paper Session 1

Collin Lynch, Kevin Ashley, Vincent Aleven, and Niels Pinkwart (Carnegie Mellon University and University of Pittsburgh): Defining Ill-Defined Domains; A literature survey

Nguyen-Thinh Le (University of Hamburg): A Constraint-based Assessment Approach for Free-Form Design of Class Diagrams using UML

Michael Heilman and Maxine Eskenazi (Carnegie Mellon University): Language Learning: Challenges for Intelligent Tutoring Systems

9.45 - 10.00 Coffee Break

10.00 - 11.30 Paper Session 2

Amy Ogan, Ruth Wylie, and Erin Walker (Carnegie Mellon University): The challenges in adapting traditional techniques for modeling student behavior in ill-defined domains

Nguyen-Thinh Le (University of Hamburg): Using Prolog Design Patterns to Support Constraint-Based Error Diagnosis in Logic Programming

Vincent Aleven, Niels Pinkwart, Kevin Ashley, and Collin Lynch (Carnegie Mellon University and University of Pittsburgh): Supporting Self-explanation of Argument Transcripts: Specific v. Generic Prompts

Amali Weerasinghe and Antonija Mitrovic (University of Canterbury): Individualizing Self-Explanation Support for Ill-Defined Tasks in Constraint-based Tutors

11.30 - 12.00 General Discussion

12.00 - 13.30 Lunch Break

13.30 - 15.00 Paper Session 3

Toby Dragon and Beverly Park Woolf (University of Massachusetts-Amherst): Guidance and Collaboration Strategies in Ill-defined Domains

Hao-Chuan Wang, Carolyn P. Rosé, Tsai-Yen Li, and Chun-Yen Chang (Carnegie Mellon University, National Chengchi University Taiwan and National Taiwan Normal University): Providing Support for Creative Group Brainstorming: Taxonomy and Technologies

Ilya Goldin, Kevin Ashley, and Rosa Pinkus (University of Pittsburgh): Teaching Case Analysis through Framing: Prospects for an ITS in an ill-defined domain

Amy Ogan, Vincent Aleven, and Christopher Jones (Carnegie Mellon University): Culture in the Classroom: Challenges for Assessment in Ill-Defined Domains

15.00 - 15.30 General Discussion


  • April 7, 2006: Submission deadline for workshop papers

  • April 25, 2006: Acceptance notification

  • May 10, 2006: Final version deadline for workshop papers

  • June 27, 2006: Workshop


Authors are asked to use the ITS workshop paper template which can be downloaded from here (MS Word) or here (LaTex).
submit your papers by email to

Organized by Vincent Aleven and Niels Pinkwart (Carnegie Mellon University), Kevin Ashley and Collin Lynch (University of Pittsburgh)