NSF Grant IIS-1320064

PI: Carolyn Penstein Rosé, Carnegie Mellon University, cprose@cs.cmu.edu , http://www.cs.cmu.edu/~cprose

Co-PI: Marcela Borge, Penn State University, mborge@psu.edu , https://www.ed.psu.edu/directory/mbs15

Submission for NSF Teaching and Learning 2015

DANCE: Discussion Affordances for Natural Collaborative Exchange

With the recent press given to online education and increasing enrollment in online courses, the need for scaling up quality educational experiences online has never been so urgent. Current offerings provide excellent materials including video lectures, exercises, and some forms of discussion opportunities. The biggest limitations are related to the human side of effective educational experiences including personal contact with instructors and the cohort experience. In the past decade, special concern has centered on students’ inability to communicate effectively, negotiate ideas, or engage in other aspects of collaborative problem-solving activity. At least a decade of research shows that students can nevertheless benefit from their interactions in learning groups when automated support is provided, especially interactive and context sensitive support.

This proposal brings together leading-edge researchers in the field of computer-supported collaborative learning with a track record for developing effective, dynamic support for collaborative learning addressing the whole learner, including cognitive, metacognitive, and social needs of learners. A key component of that work is the use of intelligent conversational agents to support productive learning processes in groups. In a nutshell the goal is to develop adaptive online learning communities that dynamically assign learners to short-term learning groups that meet their immediate learning and interaction needs, with dynamic support for productive group dynamics and in this way use the students themselves to provide much of the human side of instructional support that enables them to thrive in online learning environments, and instructor effort can be channeled efficiently where it is most needed.

The proposed CREATE project integrates an empirical foundation for supporting the cognitive, metacognitive, and social needs of online learners built up through a series of design studies, but which has never been modeled computationally, with a technological solution that has been validated through experimental studies and quantitative analyses of pre to post-test learning gains. The result will be a novel automated group learning support solution capable of meeting the needs of learning groups in a way that is more comprehensive than existing work in automated analysis of collaborative learning interactions. The proposed feasibility project investigates through experimental studies the causal effects of the behaviors of the supportive agents on student behavior, learning, and motivation. Students’ real-world regulatory activity combined with collaborative-learning theory will be used to inform the algorithms developed to trigger supportive behaviors that provide guidance or advice, and to promote process awareness for students in the chat environment. Effectiveness of support will be evaluated in terms of pre to post-test learning gains, motivational measures, and process analyses. The PIs will evaluate the scalability and sustainability of the CREATE environment through deployment in multiple classrooms and assessment of perceived usefulness and usability by teachers and students.

Intellectual Merit: The proposed work adds new knowledge to the existing work on the use of metacognitive activities to support and improve students' collaborative problem solving and learning processes through a novel approach to computational modeling and automated triggering of support. It contributes new knowledge about the effects of intelligent conversational agent behaviors on the awareness and control of group intellectual activity.

Broader impacts: Developing a reliable and valid way to guide and assess collaborative intellectual activity in online environments would be a huge asset to instructors and students both for supporting the instruction as well as for enabling large-scale assessment. The PI’s will work with The Center for Online Innovations in Learning, a partnership between the College of Education, World Campus (a highly respected online course provider), and the college of Information Sciences and Technology at Penn State. This partnership will enable the PIs to deploy and test the embedded environment in multiple courses. The PI’s will also document the study in a website and publish papers in well-known education and information-sciences journals and conferences.

• Shimoda, T., White, B., Borge, M., & Frederiksen, J. (2013). Designing for science learning and collaborative discourse. In Proceedings of the 12th International Conference on Interaction Design and Children (pp. 247-256). ACM.
• Carroll, J., Borge, M., & Shih, S. (2013). Cognitive Artifacts as a Window on Design. Cognitive Artifacts as a Window on Design. Journal of Visual Languages and Computing (2013)
• Adamson, D., Dyke, G., Jang, H. J., Rosé, C. P. (2014). Towards an Agile Approach to Adapting Dynamic Collaboration Support to Student Needs, International Journal of AI in Education 24(1), pp91-121.
• Adamson, D., Bharadwaj, A., Singh, A., Ashe, C., Yaron, D., Rosé, C. P. (2014). Predicting Student Learning from Conversational Cues, Proceedings of Intelligent Tutoring Systems
• Borge, M. (2014). Systems thinking as a design problem: A response to Litzinger and Minstrell et al. To Appear in a book edited by Richard A. Duschl
• Borge, M., Goggins, S. (2014). Developing a Community of Learners With Social Media. Proceedings of the 11th International Conference of the Learning Sciences.
• Borge, M., & Carroll, J. M. (2014). Verbal Equity, Cognitive Specialization and Performance. Proceedings of the ACM Group 2014 Conference.
• Carroll, J., Jiang, H., Borge, M. (2014). Distributed collaborative homework Activities in a problem-based usability engineering course. To Appear in the Journal of Education and Information Technologies. Measurement Approaches for Team Cognition Research, American Psychological Society.
• Hmelo-Silver, C., Rosé, C. P., Levy, J. (2014). Fostering a Learning Community in MOOCs, in Proceedings of the LAK 2014 Workshop on Conceptual Approaches to Connecting Levels of Analysis in Networked Learning.
• Rosé, C. P. (2014). Automated Linguistics Analysis as a Lens for Analysis of Group Learning, in Proceedings of the Second International Workshop on Discourse-Centric Learning Analytics.
• Yang, D., Piergallinin, M., Howley, I., & Rosé, C. P. (2014).Forum Thread Recommendation for Massive Open Online Courses, Proceedings of Educational Data Mining (short paper)
• Yang, D., Wen, M., Rosé, C. P. (2014). Peer Influence on Attrition in Massively Open Online Courses, Proceedings of Educational Data Mining (poster)
• Yang, D., Sinha, T., Adamson, D., & Rosé, C. P. (2013). Turn on, Tune in, Drop out: Anticipating student dropouts in Massive Open Online Courses, NIPS Data-Driven Education Workshop.
• Clarke, S., Resnick, L. & Rosé, C. P. (2015). Academic Dialogue, Teaching, & Learning. Invited chapter in the Handbook of Educational Psychology on Classroom Teaching.
• Rosé, C. P., Goldman, P., Sherer, J. Z., Resnick, L. (2015). Supportive Technologies for Group Discussion in MOOCs, Current Issues in Emerging eLearning, Special issue on MOOCs, January 2015.
• Yang, D., Wen., M., Rosé, C. P. (2015). Weakly Supervised Role Identification in Teamwork Interactions, Proceedings of the Association for Computational Linguistics
• Xu, W., Yang, D., Wen, M., Koedinger, K. R., & Rosé, C. P. (2015). How does student’s cognitive behavior in MOOC Discussion Forums affect Learning, Proceedings of Educational Data Mining
• Ferschke, O., Howley, I., Tomar, G., Yang, D., Rosé, C. P. (in press). Fostering Discussion across Communication Media in Massive Open Online Courses, Proceedings of Computer Supported Collaborative Learning
• Ferschke, O., Yang, D., Tomar, G., Rosé, C. P. (in press). Positive Impact of Collaborative Chat Participation in an edX MOOC, Proceedings of AI in Education (full paper)
• Howley, I., Tomar, G., Yang, D., Ferschke, O. and Rosé, C. P. (in press) Expectancy Value Theory of Help Seeking Applied to Features in MOOCs, Proceesings of AI in Education (short paper)
• Wen, M., Yang, D., and Rosé, C. P. (in press). Virtual Teams in Massive Open Online Courses, Proceesings of AI in Education (short paper)
• Aleven, V., Sewall, J., Popescu, O., Xhakaj, F., Chand, D., Baker, R., Wang, E., Siemens, G., Rosé, C. P., and Gasevic, D. (in press). Intelligent tutoring systems and MOOCs: The beginning of a beautiful friendship? Proceesings of AI in Education (short paper)
• Borge, M., Ong, Y., Rosé, C. P. (in press). Design models to Support the Development of High Quality Collaborative Reasoning in Online Settings , Proceedings of Computer Supported Collaborative Learning
• Yang, D., Wen, M., Howley, I., Kraut, R., & Rosé, C. P. (2015). Exploring the Effect of Confusion in Discussion Forums of Massive Open Online Courses, in , Proceedings of the Second ACM Conference on Learning @ Scale