Online Interventions: Summarization and Conversational Agent Technology Triggered through Automatic Discourse Analysis

In collaborative online learning environments, when two or more students are working together, a core part of learning takes place through conversation. While the online environment may exist solely as a way to bring geographically separated students together, it may also have a more important and active supporting role; for example, actually participating in student conversations, assessing the quality of student conversations, or making suggestions to the students about their conversations. Many of my current funded projects are actively making progress towards the goal of effective, dynamic support for collaborative learning. This effort builds on the linguistic analysis of collaboration work by enabling conversational support to be triggered based on an awareness of the state of the collaboration. A major aspect of this research began with my former PhD student Rohit Kumar’s development of the Basilica architecture, which facilitates rapid development of multi-party collaboration environments. A recently published journal article (Kumar & Rosé, 2011) describes a series of collaborative environments developed through this architecture using reusable components.

Until recently, the state-of-the-art in computer supported collaborative learning has consisted of static forms of support, such as structured interfaces, prompts, and assignment of students to scripted roles, all of which typically treat students in a one-size-fits-all fashion. In contrast, dynamic forms of collaboration support “listen in” on student conversations in search of important events that present opportunities for discouraging negative behavior or encouraging positive behavior using a form of text classification I refer to as automatic collaborative learning process analysis. My group is widely recognized as playing a major role in enabling this paradigm shift.

Within the Basilica architecture, interactive support agents that can participate with students in the collaborative discussion are triggered as a way of interactively offering support. A series of large scale classroom studies conducted since Fall of 2006 demonstrates the pedagogical effectiveness of this approach (Kumar et al., 2007; Wang et al., 2007; Kumar et al., 2007b; Chaudhuri et al., 2008; Chaudhuri et al., 2009; Kumar et al., 2010; Ai et al., 2010; Kumar & Rosé, 2011; Howley et al., 2011; Howley et al., 2012; Dyke et al., Under Review). In one study, students who worked with a partner with the dynamic collaborative learning support learned 1.24 standard deviations more than control condition students (Kumar et al., 2007). Students in all conditions worked in the same on-line environment. Control condition students worked alone without support. Students who either worked with a partner but without support or with support but without a partner learned 1 standard deviation more than Control condition students. Subsequent evaluations of refined versions of this automatic support have led to further improvements in effectiveness.

Subsequent studies have yielded additional insights:
  1. Personalized agents increase supportiveness and help exchange between students (Kumar et al., 2007b)
  2. Agents are more effective when students have control over timing of the interaction (Chaudhuri et al., 2008; Chaudhuri et al., 2009)
  3. Agents that employ Balesian social strategies are more effective than those that do not (Kumar et al., 2010; Ai et al., 2010)
  4. Students are sensitive to agent rhetorical strategies such as displayed bias (Ai et al., 2010), displayed openness to alternative perspectives (Kumar et al., 2011), and targeted elicitation (Howley et al., 2012)
  5. Agents that employ Accademically Productive Talk facilitation techniques significantly increase learning (Dyke et al., in press; Adamson et al., 2013)

In a partnership with Lauren Resnick at the Learning Research and Development Center, I have been investigating the role online collaborative activities can play in preparing high school students for whole class teacher lead discussions, as part of a two year professional development program in which Lauren Resnick’s team has been working directly with teachers to train them to use a classroom facilitation technique referred to as Academically Productive Talk (APT). Using technology supported analysis of classroom discussions collected over the two years, we have determined that preparing students prior to teacher lead discussions by means of online collaborative learning activities has a facilitating effect on teacher uptake of APT, increasing its prevalence by over a standard deviation in discussion sessions occurring in the class period immediately following an online activity (effect size 1.7 standard deviations).

Carolyn Penstein Rose (cprose@cs.cmu.edu)/ Carnegie Mellon University