Exploring the Learning Benefits of Erroneous Examples and Their Dynamic Adaptations Within the Context of Middle School Mathematics

Helping students to have a stronger conceptual understanding of mathematics is a major goal of K-12 instruction in the United States. Results of national and international assessments unfortunately show that while U.S. students perform adequately on applying procedures for standard computations, they have difficulty in problem solving that requires meta-cognitive strategies. Additionally, students in American schools consistently perform much worse in mathematics than their counterparts in many other countries. What can be done about the weak performance and mathematical understanding of American students?

We propose to address this issue by presenting students with problems and problem solving techniques that fall outside the classroom norm. In particular, we intend to present students with interactive erroneous examples of mathematical problems-step-by-step demonstrations of problem-solving in which one or more of the steps are incorrect. Erroneous examples are rarely used in classrooms because mathematics teachers are concerned that discussing errors may be embarrassing or make students more likely to make those errors, in behaviorist fashion. Very few empirical studies have tested the pedagogical value of erroneous examples, and no studies have explored adaptive presentation of such examples (e.g., presenting erroneous examples when a student is ready, withholding when not, providing help only when needed).

We hypothesize that the effect of erroneous examples depends on when and how they are introduced to the student: in particular, whether the errors are of others (versus self), whether students are prompted to find the error, and whether help is made available to students. We will test this hypothesis by extending an existing middle-school mathematics website (Mathtutor) with AI technology to present erroneous examples (together with correct examples and tutored exercises) and help for diagnosing the examples, in an adaptive fashion, tailoring materials to particular students' needs. Our software will prompt students to find the error(s), explain the error(s) by multiple-choice or multimedia answers, and correct the incorrect steps, among other activities. Our empirical studies will test a range of interventions and controls implemented in the software to see how the technology supports learning, motivation, and error-detection skills.

Our materials will be targeted at middle-school students in after-school programs. We have enlisted four middle schools in three states, plus an NSF-supported learning science center that works with schools. We expect that at least 200 students per school year will work with our materials and participate in our studies. Our target area is decimals, a subdomain of mathematics that is known to be difficult for middle schoolers. Our target population will be primarily 6th graders, across a range of mathematics achievement, gender, ethnicity, and English language proficiency. We will also try the materials with 7th and 8th graders who need remedial support.

Our general plan is to execute a yearly cycle of development, lab study, classroom study, and analysis. We will use log data and online assessments to measure outcomes. Each year we will investigate different aspects of erroneous examples and adaptations. We will use results of the earlier experiments to inform both new technical developments and subsequent experiments. As we develop new materials, and verify them with experimental use, we will make those materials available to the larger population of middle school students using our free website. The practical outcome of our project will be an adaptive website that can be used to support middle school students' learning of mathematics. The scientific outcome will be to pinpoint research-based design principles for making erroneous examples beneficial to a wide range of students.