Discourse Analysis

One of the major cross-cutting thrusts of my work is identification of conversational constructs that predict important individual difference variables including motivational constructs and participation goals as well as individual and group outcomes such as learning, knowledge transfer, relationship formation and trust, stress reduction, and decision making. The theoretical contribution of my work in this area is the reinterpretation of largely qualitative frameworks from sociolinguistics and discourse analysis from a computational perspective, with a particular focus on frameworks characterizing interpersonal dynamics within the theory of Systemic Functional Linguistics (SFL).

What distinguishes SFL from other theories in linguistics and makes it appropriate for achieving my research goals is its singular focus on the specification of systems of strategic linguistic choices encoded as signals situated within social contexts. This stands in contrast to generative theories of linguistics where the focus is on identification of universal language principles, which are only meant to explain aspects of linguistic structure that arise from the parameters of human cognition. It is unique as a linguistic theory that was developed by linguists working hand in hand with sociologists with the purpose of representing and explaining social processes at multiple levels, starting with the individual, and extending to pairs, small groups, communities, and even ecologies of communities acting and reacting to one another. The focus of my work is to take these rich and expansive but frequently informally specified constructs and formalize them as operationalizations that capture the most important essence. The challenge is in identifying what that most important essence is in a way that lends itself to some amount of generalization across contexts.

Computer supported collaborative learning activities have been the most frequent contexts in which my work has been situated. Foundational work reinterpreting constructs from SFL and applying them to analysis of collaborative learning interactions appears in an invited chapter on linguistic analysis of collaboration in the International Handbook of Collaborative Learning (Howley, Mayfield, & Rosé, 2013). That chapter describes a vision for a uniquely linguistic operationalization of collaborative discussion processes called SouFLé that is designed to be agnostic to specific theoretical frameworks in the learning sciences. That chapter motivates the computational reinterpretation of two specific constructs from SFL that have played a prominent role in my work, namely Martin and Rose’s Negotiation framework (Martin & Rose, 2003), which captures the ways in which participants in interactions involving the flow of knowledge and action position themselves either as sources or as recipients, and Martin & White’s Engagement framework (Martin & White, 2005), in which assertions are positioned in relation to projected speaker and audience perspectives, as well as those of third parties. A third construct included in SouFLé, but which does not have its roots in SFL, is Transactivity (Berkowitz & Gibbs, 1987), which is a form of collaborative knowledge integration widely regarded as valuable across alternative theories of collaborative learning.

The three primary component constructs of SouFLé fit together as an integrated characterization of social positioning in interaction as described in a related invited chapter, this time for the Handbook of Educational Technology (Rosé, 2012), where the framework is motivated from the perspective of assessment of the social impact of educational technology. The Negotiation dimension can be seen as describing positioning in the vertical dimension where authoritativeness is seen as related to making assertions without seeking external validation, and taking agency in action. The Heteroglossia dimension, which is derived from Martin and White’s Engagement framework, can be seen as describing positioning in the horizontal dimension where positioning an assertion in relation to the perspectives of multiple stakeholders within an interaction has implications for solidarity between participants. Transactivity relates to a third dimension of forward motion within a conversation, where contributions may introduce new directions, and therefore push the conversation forward into new territory, or build on earlier contributions, and therefore enrich the representation of an existing focus of the conversation.

Martin & Rose’s Negotiation framework has a prior history in analysis of interaction in learning contexts beginning with work on classroom facilitation (Veel, 1999) where it was used to highlight systematic differences in style between teachers that maintain a position as the sole source of information in class discussions and teachers who productively engage students with one another in discussions that center on student reasoning. Our operationalization allows us to systematically identify meaningful threads within complex interactions (Mayfield et al., 2012a) and to assign a rating to each participant that we refer to as an Authoritativeness score for a participant (Mayfield et al., 2011; Howley et al., 2011), which refers to the percentage of threads in which the participant played the role of source. In reducing the pattern of codes to a scale, we are then able to examine the extent to which positioning on the vertical dimension correlates with extra-linguistic variables. We expect to see positive correlations between Authoritativeness and extra-linguistic variables that are associated with a value placed on capability in connection with the specific knowledge and action associated with the threads used in the computation. Application of the same coding scheme to data in strikingly different contexts challenges an overly simplistic interpretation of the significance of the Authoritativeness rating.

For example, Authoritativeness correlates both with domain related academic self-efficacy and learning in collaborative problem solving settings (Howley et al., 2011; Howley et al., 2013), which makes sense since the ability to provide knowledge and act in task relevant ways is what is academic self-efficacy measures in these contexts, and the tasks are designed in such a way that meaningful task engagement is meant to produce learning. What is even more interesting is that it also sheds light on the interplay between social and cognitive factors in learning, and points to opportunities for impacting engagement in important learning behaviors by addressing social problems such as bullying (Cui et al., 2008; Howley et al., 2013).

It is consistent with this interpretation to expect different correlations in contexts where the expectations associated with task roles are different, such as in doctor-patient interactions where the doctor is expected to have special knowledge not possessed by the patient. As an evaluation of the predictive validity of our Authoritativeness metric in a health context, in the past year we have applied the Authoritativeness metric to analysis of doctor-patient communication (Mayfield et al., Under Review). We measured the predictive validity of this metric in connection with validated measures related to trust in doctor-patient communication. In particular, we tested 5 specific trust related constructs selected by colleagues at Brown university who specialize in trust in doctor-patient communication. We determined that over a corpus of 450 doctor-patient interactions paired with questionnaire data, 4 out of 5 constructs were significantly correlated with Authoritativeness, with R values ranging from .25 to .35 using Authoritativeness scores that were computed from hand coded Negotiation codes. A construct related to patient health efficacy from the same questionnaire data did not correlate with patient Authoritativeness, which is expected in this context since the role of patient comes with different expectations with respect to expertise than a collaborative problem solving session.

In addition to providing the basis for the Authoritativeness scale, the Negotiation codes more generally have been valuable for structuring multi-threaded conversational interactions in preparation for subsequent analysis, for example, analysis of task relevant differences in information sharing practices between military and civilian pairs performing the same task in a lab study (Mayfield et al., 2012b) as well as conversational strategies associated with stress reduction in online cancer support chats (Mayfield et al., 2012a; Mayfield et al., 2012c).

Martin & White’s expansive Engagement framework (Martin and White, 2005) is based in qualitative work from the field of rhetoric, where clues about projected attitudes about speaker perspective, hearer perspective, and the relationship between the two are analyzed in terms of signals included in the framing of assertions. While the assumed significance of the framing within their work was related to the idea of projecting expectations about horizontal social positions, the effects of the associated signals were never tested empirically. As with the work on Negotiation, our approach has been to pair down the original formulation, which suffered from the same issues as the original Negotiation framework, into a simple set of contrasts that could be precisely defined as a set of exhaustive codes and reliably coded. Empirical work using our operationalization confirms the effect of signals hypothesized from within the theory. In correlational analyses of collaborative learning groups, we find that students contribute significantly more reasoning oriented contributions in groups where partners adopt a style indicating openness to alternative perspectives (Dyke et al., in press). In an experimental study we find that student groups contribute significantly more design ideas in a collaborative power plant design task when a supportive conversational agent presents directions in that style rather than a style that does not project openness (Kumar et al., 2011). My work examining the impact on learning in collaborative inquiry settings where a productive learning activity is making reasoning and ideas explicit demonstrates the positive impact of increasing idea contribution in those contexts (Wang et al., 2011).

The SouFLé operationalization of Transactivity, which has been applied in multiple collaborative problem solving and collaborative design settings, breaks the construct down into two simpler constructs, one that specifies whether a contribution to a conversation makes reasoning explicit through the use of some indicator of causality, and another that indicates whether that reasoning relates back to an earlier articulation of reasoning or begins a new direction of ideation within an interaction (Gweon et al., 2011a; Gweon et al., 2011b; Gweon et al., 2012). Our work on analysis of Transactivity in connection with learning is consistent with prior work (Joshi & Rosé, 2007). Beyond its usual role as a mediating variable related to socio-cognitive conflict and learning, in a lab study representing an assembly line task we have confirmed that it is also associated with effective knowledge sharing when newcomers join a new working group (Gweon et al., 2011).

The vision for cross-theory discussion enabled in part by Souflé was realized in a series of workshops and a symposium co-organized by myself, Daniel Suthers (University of Hawai'i at Manoa), Kristine Lund (Ecole Normale Supérieure de Lyon), Christopher Teplovs (Problemshift, Inc), and Nancy Law (University of Hong Kong), and resulting in an edited volume (Suthers, Lund, Rosé, Teplovs, & Law, in press) under contract with Springer in which we have worked along with about forty other colleagues internationally who represent a variety of theoretical perspectives in the learning sciences to develop a new paradigm for analysis of collaborative learning interactions we refer to as multi-vocal analysis. Multi-vocal analysis is an iterative analytic process in which representatives of multiple theoretical and methodological perspectives have the opportunity to speak to one another and challenge one another while examining data that are of common interest. It is a step beyond a multi-methods approach because it involves analysts of opposing viewpoints working together rather than individual analysts or teams applying multiple methodologies from a common perspective. The paradigm is for at least three teams from diverse theoretical and/or methodological perspectives to independently analyze a data set from their own perspective. After the individual analyses are presented, a discussant identifies common themes and points of divergence. Group discussion then follows, after which the analysts revisit their analyses. The edited volume includes multi-vocal analyses of five data sets, which is just a subset of the data examined over the series of workshops. Out of the five data sets examined at length in the book, two of them include Souflé analyses. Even more important than the ways in which this multi-vocal process sharpened and enriched the analyses of each data set, as well documented within the volume, the community of researchers participating in the process benefitted from the intensive exchange between subcommunities that are frequently more isolated from one another in practice.

Carolyn Penstein Rose (cprose@cs.cmu.edu)/ Carnegie Mellon University