Project LISTEN
A Reading Tutor that Listens
Last updated: February 9, 2006

Summary
Awards 
In the News 
Progress 

 
Research Basis 
Publications

Photos
Videos
People

Research Basis for Project LISTEN’s Reading Tutor

 

Jack Mostow, Director, Project LISTEN

Carnegie Mellon University School of Computer Science

RI-NSH 4213, 5000 Forbes Avenue, Pittsburgh, PA 15213-3890

www.cs.cmu.edu/~listen

November 11, 2003; revised June 6, 2005

Abstract

This document summarizes two types of published research underlying Project LISTEN’s automated Reading Tutor.  Intervention studies measured the Reading Tutor’s effectiveness.  Other research, others’ as well as our own, served to guide its development.  The cited Project LISTEN publications can be downloaded from www.cs.cmu.edu/~listen except where precluded by copyright or not yet in print.

Acknowledgements

Project LISTEN is supported by NSF under ITR/IERI Grant REC-0326153, and by the Heinz Endowments.  Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or the Heinz Endowments.

Special thanks to the co-principal investigators who helped formulate the IERI proposals from which portions of this document are excerpted, especially reading expert Professor Rollanda O’Connor.

1. Summary of intervention studies

Speech-recognition-based, computer-guided oral reading has demonstrated usability, user acceptance, assistive effectiveness, and even pre- to post-test gains (Cole et al., 1999; Mostow, Roth, Hauptmann, & Kane, 1994; Nix, Fairweather, & Adams, 1998; Russell et al., 1996; Williams, 2002; Williams, Nix, & Fairweather, 2000) – but the proof of the pudding is whether it significantly increases learning gains over gains that children make otherwise.  Even with barely 20 minutes of use per day, successive versions of the Reading Tutor have produced substantially higher comprehension gains than current practices in controlled studies lasting several months. To ensure that results were due to the Reading Tutor intervention, we compared different treatments within the same classrooms and randomized treatment assignment, stratifying by pretest scores within class.   We used valid and reliable measures (Woodcock, 1998) to measure gains from pre- to post-test.  We computed effect size as the difference in gains between the Reading Tutor and current practice, divided by the average standard deviation in gains of the two groups.  Effect sizes for passage comprehension were substantial compared to other studies (NRP, 2000):  0.60 for 63 students in grades 2, 4, and 5 at a low-income urban school (Mostow & Aist, 2001; J. Mostow et al., 2003); 0.48 for 66 third graders at a lower-middle class urban school (Aist et al., 2001; Mostow et al., 2001; Jack Mostow, Greg Aist et al., 2003); and 0.66 for 52 first graders at two suburban Blue Ribbon Schools of Excellence (Mostow, Aist, Bey et al., 2002; Mostow, Beck, & others, under revision).

1.1. Pilot study (1996-97)

The Reading Tutor achieved dramatic results in the first pilot study of extended use long enough to demonstrate significant learning.  During the 1996-97 school year, a pilot group of low-reading third graders used the Reading Tutor one at a time in a small office under the individual supervision of a school aide.  According to school-administered pre- and post-tests, six third graders who started almost three years below grade level averaged two years of progress in under eight months use (Aist & Mostow, 1997).

1.2. Within-classroom comparison (1998)

In spring 1998, we did our first controlled study of the Reading Tutor in classroom settings at Fort Pitt Elementary (J. Mostow et al., 2003).  All 72 students in 3 classrooms (grades 2, 4, and 5) that had not previously used the Reading Tutor were independently pre-tested on the Word Attack, Word Identification, and Passage Comprehension subtests of the Woodcock Reading Mastery Test (Woodcock, 1987).  We split each class into 3 matched treatment groups – Reading Tutor, commercial reading software, or regular classroom activities, including other software use.  We assigned students to treatments randomly, matched within classroom by pretest scores.  Even though the study lasted only 4 months, and actual usage was a fraction of the planned daily 20-25 minutes, students who used the 1998 version of the Reading Tutor significantly outgained their matched classmates in comprehension (effect size .60, p = .002), progressing faster than their national cohort.  (No other differences were significant, and commercial software fell in between.)  As the principal said, “these children were closing the gap.”

1.3. Comparison to human tutors (1999-2000)

In 1999-2000, we evaluated the new, mixed story choice version of the Reading Tutor at a second school in a lower-middle class community near Pittsburgh.  This year-long study of 131 second and third graders in 12 classrooms compared three daily 20-minute treatments. (a) 58 students in 6 classrooms used the 1999-2000 version of the Reading Tutor.  Students took daily turns using one shared Reading Tutor in their classroom while the rest of their class received regular instruction.  (b) 34 students in the other 6 classrooms were pulled out daily for one-on-one tutoring by certified teachers.  To control for materials, the human tutors used the same set of stories as the Reading Tutor.  (c) 39 students served as in-classroom controls, receiving regular instruction without tutoring.  We pre- and post-tested students in word identification, word attack, word comprehension, passage comprehension, and fluency.

To our surprise, human tutors beat the Reading Tutor only in Word Attack (effect size .55).  Third graders in both the computer- and human-tutored conditions outgained the control group in Word Comprehension (effect sizes of .56 and .72, respectively) and Passage Comprehension (effect sizes of .48 and .55, respectively) (Aist et al., 2001; Mostow et al., 2001).  No other differences in gains were significant.

1.4. Equal-time comparison to Sustained Silent Reading (2000-2001)

According to the National Reading Panel, “the amount of gain attributable to reading alone should be the baseline comparison against which the efficacy of instructional procedures is tested. If an instructional method does better than reading alone, it would be safe to conclude that method works” (NRP, 2000, Ch. 3, p. 27).  A 7-month study of 178 students in grades 1-4 at two Blue Ribbon Schools of Excellence compared two treatments, each provided in daily 20-minute sessions.  88 students did Sustained Silent Reading (SSR) as already implemented in their classrooms (including teacher read-aloud in grade 1 until students were ready for independent reading practice).  90 students in 10-computer labs used the 2000-2001 version of Project LISTEN’s Reading Tutor.  The Reading Tutor group significantly outgained their statistically matched SSR classmates in phonemic awareness, rapid letter naming, word identification, word comprehension, passage comprehension, fluency, and spelling – especially in grade 1, where effect sizes for between-treatment differences in gains ranged from .20 to .72 (Mostow, Aist, Bey et al., 2002).

1.5. Effectiveness for English language learners (2004) [this section added 6/6/05]

2004 marked the first independent, third-party, controlled evaluation of the Reading Tutor (Poulsen, 2004).  This two-month pilot study included 34 second through fourth grade Hispanic students from four bilingual education classrooms. The study compared the efficacy of the 2004 version of the Project LISTEN Reading Tutor against the standard practice of Sustained Silent Reading (SSR).  This study was undertaken to obtain some initial indication as to whether the tutor would also be effective within a population of English language learners. 

The study employed a crossover design where each participant spent one month in each of the treatment conditions.  The experimental treatment consisted of 25 minutes per day using the Reading Tutor within a small pullout lab setting.  Students in the control treatment remained in the classroom where they participated in established reading instruction activities.  Dependent variables consisted of the school district’s curriculum based measures for fluency, sight word recognition, and comprehension.

The Reading Tutor group outgained the control group in every measure during both halves of the crossover experiment.  Within-subject results from a paired T-Test indicate that these gains were significant for one sight word measure (p = .056) and both fluency measures (p < .001).  Effect sizes were 0.55 for timed sight words, a robust 1.16 for total fluency and an even larger 1.27 for fluency controlled for word accuracy.  These dramatic results observed during a one-month treatment indicate that this technology may have much to offer English language learners.

Two additional groups of Canadian researchers conducted independent evaluations of the Reading Tutor with English language learners and as of June 2005 are analyzing the data.

A 10-week study by Kenneth Reeder, Margaret Early, Maureen Kendrick, Jon Shapiro, and Jane Wakefield at the University of British Columbia (D’Silva et al., 2005) involved 77 students from five Vancouver elementary schools, grades 2-6 (ages 7-12 years).  Their home languages were Hindi (14), Mandarin (21), Spanish (21), and English (21: 11 using the Reading Tutor, and 10 in a human tutoring program).  Gains by the Reading Tutor group matched gains by the human tutoring group on most reading measures, and interviews showed favorable affect impact by the Reading Tutor.  Analysis is continuing.

A 12-week study by Esther Geva and Todd Cunningham at the University of Toronto (Cunningham & Geva, 2005) involved 104 ESL students in grades 4-6 at eight schools.  The study compared three treatments:  the Reading Tutor; Kurzweil 3000, which reads aloud to the student and provides vocabulary support; and regular ESL classroom instruction.  Analysis of data from the first 39 students shows promising trends.

2. Summary of underlying research

Why does the Reading Tutor improve comprehension?  Theoretically, students who recognize words effortlessly can devote more attention to comprehension (LaBerge & Samuels, 1974), and the relationship between rate of oral reading and reading comprehension is strong through the elementary years (Pinnel et al., 1995).   The cognitive load imposed by word identification before it has become a mentally automatic process consumes limited mental resources, such as attention and short term memory, needed to comprehend the sentence and its relationship to the surrounding context (Perfetti, 1992). 

However, decoding practice by itself does not necessarily improve fluency or comprehension.  Some studies found that teaching children to recognize isolated words quickly gave no advantage in reading comprehension (Fleischer, Jenkins, & Pany, 1979), or that comprehension did not improve unless readers recognized the words nearly as fast in context as in lists (Levy, Abello, & Lysynchuk, 1997).  Thus fluency makes a unique contribution to comprehension over that made by word identification (Ehri & McCormick, 1998; O'Connor et al., 2002; Shankweiler et al., 1999). 

Guided oral reading provides opportunities to practice word identification and comprehension in context.  There is ample evidence that one of the major differences between good and poor readers is the amount of time they spend reading.  Poor readers are unlikely to practice on their own.  Students who need the most practice spend the least amount of time actually reading (Allington, 1977).  How time is spent reading matters too (Mostow, Aist, Beck et al., 2002).  Poor readers tend to reread the same easy stories over and over (Aist, 2002a).  Modifying the Reading Tutor to take turns picking stories exposed students to more new vocabulary than they saw when they chose the stories (Aist, 2002a, 2002b; Aist & Mostow, 2003; J. Mostow et al., 2003).

The Reading Tutor aims for the zone of proximal development (Doolittle, 1997) by dynamically updating its estimate of the student’s reading level, and picking stories accordingly – which are somewhat harder than students choose when it is their turn (Jack Mostow, Greg Aist et al., 2003).

The Reading Tutor scaffolds key processes in reading – and tests its own scaffolding.  Scaffolding provides information at the “teachable moments” when it is needed.  For example, explicit vocabulary instruction is important but time-consuming (Beck, McKeown, & Kucan, 2002).  Explaining unfamiliar words and concepts in context can remediate deficits in vocabulary and background knowledge (Elley, 1989), so we added support for vocabulary acquisition by presenting short “factoids” – comparisons to other words (Aist, 2001b, 2002a).  An automated experiment embedded in the Reading Tutor tested the effectiveness of reading a factoid just before a new word in a story, compared to simply encountering the word in context without a factoid.  The outcome variable was performance on a multiple-choice question, presented the next day the student used the Reading Tutor.  Analysis of over 3,000 randomized trials showed that factoids helped on rare, single-sense words, and that they helped third graders more than second graders (Aist, 2000, 2001a, 2001b). By acquiring predictive models of the effects of tutorial actions, embedded experiments can inform a decision-theoretic approach to tutoring (Beck, 2001, 2002; Beck & Woolf, 2000, 2001; Beck, Woolf, & Beal, 2000; Murray, VanLehn, & Mostow, revisions under review).

The zone of proximal development depends on tutorial scaffolding as well as on student proficiency (Murray & Arroyo, 2002), so the Reading Tutor lets the student read as much as possible, but helps as much as necessary.  It provides spoken and graphical assistance when it notices the student click for help, hesitate, get stuck, skip a word, make a mistake, or encounter a word likely to be misread (Mostow & Aist, 1999).  Its “visual speech” (Massaro, 1998) uses talking-mouth videoclips of phonemes to scaffold phonemic awareness.  The Reading Tutor assists word identification by previewing new words (Mostow, to appear) and reading hard words aloud.  Its word attack hints include rhyming and sounding out.  It supports vocabulary acquisition by explaining new words (Aist, 2001b, 2002a; Jack Mostow, Joseph Beck et al., 2003).  It scaffolds comprehension by reading hard sentences aloud and by asking questions (NRP, 2000) – “cloze” items (Mostow, Tobin, & Cuneo, 2002) and generic “who-what-where” questions, which at first appeared to boost comprehension of nearby sentences in an embedded experiment (Beck, Mostow, Cuneo, & Bey, 2003).  The Reading Tutor bolsters motivation by listening attentively, “backchanneling(Aist & Mostow, 1999), giving encouragement (Aist, Kort, Reilly, Mostow, & Picard, 2002), and praising good or improved performance (Mostow & Aist, 1999). By reducing frustration (Betts, 1946) and making a wide range of authentic, engaging text cognitively accessible to the child, scaffolding helps address the motivational issues of confidence, challenge, curiosity, and control pivotal to effective tutoring (Lepper & Chabay, 1988; Lepper, Woolverton, Mumme, & Gurtner, 1993).  Poor readers’ listening comprehension is far above their independent reading level (Curtis, 1980; Spache, 1981), so reading hard words and sentences to them reduces frustration and repairs comprehension failures caused by lack of automaticity in word identification.

One approach to improving automaticity is repeated reading, in which students read a passage or page of text until their reading rate increases by a given amount, usually 25% or more (Samuels, 1979).  A recent review of the repeated reading literature (Meyer & Felton, 1999) recommended that poor readers practice building fluency for 10-20 minutes per day over a long duration, engage in reading aloud, and use text at their instructional level. However, improving word recognition accuracy and comprehension can require assistance to remediate errors (McCoy & Pany, 1986; Young, Bowers, & MacKinnon, 1996) – which requires listening to the student read aloud.

References (download Project LISTEN publications at www.cs.cmu.edu/~listen)

Aist, G. (2000). Helping Children Learn Vocabulary during Computer-Assisted Oral Reading. Unpublished Ph.D. dissertation, Carnegie Mellon University, Pittsburgh, PA.

Aist, G. (2001a). Factoids: Automatically constructing and administering vocabulary assistance and assessment. In J. D. Moore & C. L. Redfield & W. L. Johnson (Eds.), Artificial Intelligence in Education:  AI-ED in the Wired and Wireless Future (pp. 234-245). San Antonio, Texas: Amsterdam:  IOS Press.

Aist, G. (2001b). Towards automatic glossarization:  Automatically constructing and administering vocabulary assistance factoids and multiple-choice assessment. International Journal of Artificial Intelligence in Education, 12, 212-231.

Aist, G. (2002a). Helping Children Learn Vocabulary During Computer-Assisted Oral Reading. Educational Technology and Society, 5(2), http://ifets.ieee.org/periodical/vol_2_2002/aist.html.

Aist, G. (2002b, April 29). Helping Children Learn Vocabulary during Computer-Assisted Oral Reading:  A Dissertation Summary [Poster presented as a Distinguished Finalist for the Outstanding Dissertation of the Year Award]. 47th Annual Convention of the International Reading Association, San Francisco, CA

Aist, G., Kort, B., Reilly, R., Mostow, J., & Picard, R. (2002, October 14-16). Experimentally Augmenting an Intelligent Tutoring System with Human-Supplied Capabilities:  Adding Human-Provided Emotional Scaffolding to an Automated Reading Tutor that Listens. Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces (ICMI 2002), Pittsburgh, PA, 483-490.

Aist, G., & Mostow, J. (1997, October). When Speech Input is Not an Afterthought: A Reading Tutor that Listens. Workshop on Perceptual User Interfaces, Banff, Canada

Aist, G., & Mostow, J. (1999, September). Measuring the Effects of Backchanneling in Computerized Oral Reading Tutoring. Proceedings of the ESCA Workshop on Prosody and Dialog, Eindhoven, Netherlands

Aist, G., & Mostow, J. (2003). Faster, better task choice in a reading tutor that listens. In F. N. Fisher (Ed.), Speech Technology for Language Learning. Lisse, The Netherlands: Swets & Zeitlinger Publishers.

Aist, G., Mostow, J., Tobin, B., Burkhead, P., Corbett, A., Cuneo, A., Junker, B., & Sklar, M. B. (2001). Computer-assisted oral reading helps third graders learn vocabulary better than a classroom control – about as well as one-on-one human-assisted oral reading. In J. D. Moore & C. L. Redfield & W. L. Johnson (Eds.), Artificial Intelligence in Education:  AI-ED in the Wired and Wireless Future (pp. 267-277). San Antonio, Texas: Amsterdam:  IOS Press.

Allington, R. (1977). If they don't read much, how they ever gonna get good? Journal of Reading, 21, 57-61.

Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing Words to Life:  Robust Vocabulary Instruction. NY: Guilford.

Beck, J. E. (2001). ADVISOR:  A Machine-Learning Architecture for Intelligent Tutor Construction. Unpublished Ph.D., University of Massachusetts Amherst, Amherst.

Beck, J. E. (2002). Directing Development Effort with Simulated Students. Proceedings of the Sixth International Conference on Intelligent Tutoring Systems, Biarritz, France, 851-860.

Beck, J. E., Mostow, J., Cuneo, A., & Bey, J. (2003, July 20-24). Can automated questioning help children's reading comprehension? Proceedings of the Tenth International Conference on Artificial Intelligence in Education (AIED2003), Sydney, Australia, 380-382.

Beck, J. E., & Woolf, B. P. (2000). High-level Student Modeling with Machine Learning. Fifth International Conference on Intelligent Tutoring Systems, 584-593.

Beck, J. E., & Woolf, B. P. (2001). Using Rollouts to induce a policy from a user model. Eighth International Conference on User Modeling, 210-212.

Beck, J. E., Woolf, B. P., & Beal, C. R. (2000). ADVISOR: A machine learning architecture for intelligent tutor construction. Proceedings of the Seventeenth National Conference on Artificial Intelligence, Austin, Texas, 552-557.

Betts, E. A. (1946). Foundations of Reading Instruction. New York: American Book Company.

Cole, R., Massaro, D. W., Villiers, J. d., Rundle, B., Shobaki, K., Wouters, J., Cohen, M., Beskow, J., Stone, P., Connors, P., Tarachow, A., & Solcher, D. (1999, April). New tools for interactive speech and language training: Using animated conversational agents in the classrooms of profoundly deaf children. ESCA/SOCRATES Workshop on Method and Tool Innovations for Speech Science Education, London, UK

Cunningham, T., & Geva, E. (2005, June 24). The effects of reading technologes on literacy development of ESL students [poster presentation]. Twelfth Annual Meeting of the Society for the Scientific Study of Reading, Toronto

Curtis, M. E. (1980). Development of components of reading skill. Journal of Educational Psychology, 72(5), 656-669.

D’Silva, R., Hong, L., Lenters, K., Early, M., Kendrick, M., Reeder, K., Shapiro, J., & Wakefield, J. (2005, May). LISTENing to diverse learners: The effectiveness and appropriateness of a computer-based reading tutor for young multilingual learners [poster presentation]. Fifth Faculty of Education Research Day, The University of British Columbia, Vancouver, BC

Doolittle, P. E. (1997). Vygotsky's Zone of Proximal Development as a Theoretical Foundation for Cooperative Learning. Journal on Excellence in College Teaching, 8(1), 83-103.

Ehri, L., & McCormick. (1998). Phases of word learning: Implications for instruction with delayed and disabled readers. Reading and Writing Quarterly, 14, 135-164.

Elley, W. B. (1989). Vocabulary acquisition from listening to stories. Reading Research Quarterly, 24, 174-187.

Fleischer, L. S., Jenkins, J. R., & Pany, D. (1979). Effects on poor readers' comprehension of training in rapid decoding. Reading Research Quarterly, 14, 30-48.

LaBerge, D., & Samuels, J. (1974). Toward a theory of automatic information processing in reading. Cognitive Psychology, 6, 293-323.

Lepper, M. R., & Chabay, R. W. (1988). Socializing the intelligent tutor: Bringing empathy to computer tutors. In H. Mandl & A. Lesgold (Eds.), Learning issues for intelligent tutoring systems (pp. 242-257). NY: Springer.

Lepper, M. R., Woolverton, M., Mumme, D. L., & Gurtner, J. (1993). Motivational Techniques of Expert Human Tutors:  Lessons for the Design of Computer-Based Tutors. In S. P. Lajoie & S. J. Derry (Eds.), Computers as Cognitive Tools (pp. 75-105). Hillsdale, NJ: Erlbaum.

Levy, B., Abello, B., & Lysynchuk, L. (1997). Transfer from word training to reading in context: Gains in reading fluency and comprehension. Learning Disabilities Quarterly, 20, 173-188.

Massaro, D. (1998). Perceiving Talking Faces. Cambridge, MA: MIT Press.

McCoy, K., & Pany, D. (1986). Summary and analysis of oral reading corrective feedback research. The Reading Teacher, 39, 548-555.

Meyer, M. S., & Felton, R. H. (1999). Repeated reading to enhance fluency: Old approaches and new directions. Annals of Dyslexia, 49, 283-306.

Mostow, J. (to appear). Evaluation purposes, excuses, and methods:  Experience from a Reading Tutor that listens. In L. Verhoeven (Ed.), Facilitating Constructivist Literacy Environments Through Technology. Mahway, NJ: Erlbaum.

Mostow, J., & Aist, G. (1999). Giving help and praise in a reading tutor with imperfect listening -- because automated speech recognition means never being able to say you're certain. CALICO Journal, 16(3), 407-424.

Mostow, J., & Aist, G. (2001). Evaluating tutors that listen: An overview of Project LISTEN. In K. Forbus & P. Feltovich (Eds.), Smart Machines in Education (pp. 169-234). Menlo Park, CA: MIT/AAAI Press.

Mostow, J., Aist, G., Beck, J., Chalasani, R., Cuneo, A., Jia, P., & Kadaru, K. (2002, June 5-7). A La Recherche du Temps Perdu, or As Time Goes By: Where does the time go in a Reading Tutor that listens? Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (ITS'2002), Biarritz, France, 320-329.

Mostow, J., Aist, G., Bey, J., Burkhead, P., Cuneo, A., Junker, B., Rossbach, S., Tobin, B., Valeri, J., & Wilson, S. (2002, June 27-30). Independent practice versus computer-guided oral reading: Equal-time comparison of sustained silent reading to an automated reading tutor that listens. Ninth Annual Meeting of the Society for the Scientific Study of Reading, Chicago, Illinois

Mostow, J., Aist, G., Burkhead, P., Corbett, A., Cuneo, A., Eitelman, S., Huang, C., Junker, B., Platz, C., Sklar, M. B., & Tobin, B. (2001). A controlled evaluation of computer- versus human-assisted oral reading. In J. D. Moore & C. L. Redfield & W. L. Johnson (Eds.), Artificial Intelligence in Education:  AI-ED in the Wired and Wireless Future (pp. 586-588). San Antonio, Texas: Amsterdam:  IOS Press.

Mostow, J., Aist, G., Burkhead, P., Corbett, A., Cuneo, A., Eitelman, S., Huang, C., Junker, B., Sklar, M. B., & Tobin, B. (2003). Evaluation of an automated Reading Tutor that listens:  Comparison to human tutoring and classroom instruction. Journal of Educational Computing Research, 29(1), 61-117.

Mostow, J., Aist, G., Huang, C., Junker, B., Kennedy, R., Lan, H., Latimer, D., O'Connor, R., Tassone, R., Tobin, B., & Wierman, A. (2003). 4-Month Evaluation of a Learner-controlled Reading Tutor that Listens. In F. N. Fisher (Ed.), Speech Technology for Language Learning. Lisse, The Netherlands: Swets & Zeitlinger Publishers.

Mostow, J., Beck, J., Bey, J., Cuneo, A., Sison, J., & Tobin, B. (2003, June 12-15). An Embedded Experiment to Evaluate the Effectiveness of Vocabulary Previews in an Automated Reading Tutor. Tenth Annual Meeting of the Society for Scientific Studies of Reading, Boulder, CO

Mostow, J., Beck, J., & others. (under revision). Computer-Guided Oral Reading versus Independent Practice:  Comparison of Sustained Silent Reading to an Automated Reading Tutor that Listens. Journal of Educational Psychology.

Mostow, J., Roth, S. F., Hauptmann, A. G., & Kane, M. (1994, August). A prototype reading coach that listens [AAAI-94 Outstanding Paper Award]. Proceedings of the Twelfth National Conference on Artificial Intelligence, Seattle, WA, 785-792.

Mostow, J., Tobin, B., & Cuneo, A. (2002, June 3). Automated comprehension assessment in a reading tutor. Proceedings of the ITS 2002 Workshop on Creating Valid Diagnostic Assessments, San Sebastian, Spain, 52-63.

Murray, R. C., VanLehn, K., & Mostow, J. (revisions under review). Looking Ahead to Select Tutorial Actions: A Decision-Theoretic Approach. International Journal of Artificial Intelligence in Education.

Murray, T., & Arroyo, I. (2002). Towards Measuring and Maintaining the Zone of Proximal Development in Adaptive Instructional Systems. Sixth International Conference on Intelligent Tutoring Systems, 749-758.

Nix, D., Fairweather, P., & Adams, B. (1998). Speech Recognition, Children, and Reading. CHI 98, 245-246.

NRP. (2000). Report of the National Reading Panel. Teaching children to read:  An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (00-4769). Washington, DC: National Institute of Child Health & Human Development.

O'Connor, R. E., Bell, K. M., Harty, K. R., Larkin, L. K., Sackor, S., & Zigmond, N. (2002). Teaching Reading to Poor Readers in the Intermediate Grades: A Comparison of Text Difficulty. Journal of Educational Psychology, 94, 474-485.

Perfetti, C. A. (1992). The representation problem in reading acquisition. In P. B. Gough & L. C. Ehri & R. Treiman (Eds.), Reading Acquisition (pp. 145-174). Hillsdale, NJ: Lawrence Erlbaum.

Pinnel, G. S., Pikulski, J. J., Wixson, K. K., Campbell, J. R., Gough, P. B., & Beatty, A. S. (1995). Listening to children read aloud. Washington, DC: Office of Educational Research and Improvement, U.S. Department of Education.

Poulsen, R. (2004). Tutoring Bilingual Students With an Automated Reading Tutor That Listens:  Results of a Two-Month Pilot Study. Unpublished Masters Thesis, DePaul University, Chicago, IL.

Russell, M., Brown, C., Skilling, A., Series, R., Wallace, J., Bonham, B., & Barker, P. (1996). Applications of automatic speech recognition to speech and language development in young children. Proceedings of the Fourth International Conference on Spoken Language Processing, Philadelphia PA

Samuels, S. J. (1979). The method of repeated readings. The Reading Teacher, 32, 403-408.

Shankweiler, D., Lundquist, E., Katz, L., Stuebing, K. K., Fletcher, J. M., Brady, S., Fowler, A., Dreyer, L., Marchione, K. E., Shaywitz, S. E., & Shaywitz, B. A. (1999). Comprehension and decoding: Patterns of association in children with reading difficulties. Scientific Studies of Reading, 3, 69-94.

Spache, G. D. (1981). Diagnostic Reading Scales. Monterey, CA: McGraw-Hill.

Williams, S. M. (2002). Speech recognition technology and the assessment of beginning readers. In NRC (Ed.), Technology and assessment:  Thinking ahead:  Proceedings of a workshop (pp. 40-49). Board on Testing and Assessment, Center for Education, Division of Behavioral and Social Sciences and Education.  Washington, DC: National Academy Press.

Williams, S. M., Nix, D., & Fairweather, P. (2000). Using Speech Recognition Technology to Enhance Literacy Instruction for Emerging Readers. Fourth International Conference of the Learning Sciences, 115-120.

Woodcock, R. W. (1987). Woodcock Reading Mastery Tests-Revised. Circle Pines, MN: American Guidance Service.

Woodcock, R. W. (1998). Woodcock Reading Mastery Tests - Revised (WRMT-R/NU). Circle Pines, Minnesota: American Guidance Service.

Young, A. R., Bowers, P. G., & MacKinnon, G. E. (1996). Effects of prosodic modeling and repeated reading on poor readers' fluency and comprehension. Applied Psycholinguistics, 17, 59-84.