Focused or stuck together: Multimodal patterns reveal triads' performance in collaborative problem solving

Hana Vrzakova, Mary Jean Amon, Angela Stewart, Nicholas D. Duran, Sidney K. D'Mello

Research output: Chapter in Book/Report/Conference proceedingConference contribution

41 Scopus citations

Abstract

Collaborative problem solving (CPS) in virtual environments is an increasingly important context of 21st century learning. However, our understanding of this complex and dynamic phenomenon is still limited. Here, we examine unimodal primitives (activity on the screen, speech, and body movements), and their multimodal combinations during remote CPS. We analyze two datasets where 116 triads collaboratively engaged in a challenging visual programming task using video conferencing software. We investigate how UI-interactions, behavioral primitives, and multimodal patterns were associated with teams' subjective and objective performance outcomes. We found that idling with limited speech (i.e., silence or backchannel feedback only) and without movement was negatively correlated with task performance and with participants' subjective perceptions of the collaboration. However, being silent and focused during solution execution was positively correlated with task performance. Results illustrate that in some cases, multimodal patterns improved the predictions and improved explanatory power over the unimodal primitives. We discuss how the findings can inform the design of real-time interventions for remote CPS.

Original languageEnglish (US)
Title of host publicationLAK 2020 Conference Proceedings - Celebrating 10 years of LAK
Subtitle of host publicationShaping the Future of the Field - 10th International Conference on Learning Analytics and Knowledge
PublisherAssociation for Computing Machinery
Pages295-304
Number of pages10
ISBN (Electronic)9781450377126
DOIs
StatePublished - Mar 23 2020
Event10th International Conference on Learning Analytics and Knowledge: Shaping the Future of the Field, LAK 2020 - Frankfurt, Germany
Duration: Mar 23 2020Mar 27 2020

Publication series

NameACM International Conference Proceeding Series

Conference

Conference10th International Conference on Learning Analytics and Knowledge: Shaping the Future of the Field, LAK 2020
Country/TerritoryGermany
CityFrankfurt
Period3/23/203/27/20

Keywords

  • CSCL
  • CSCW
  • Interpretability
  • Multimodal Learning Analytics

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Focused or stuck together: Multimodal patterns reveal triads' performance in collaborative problem solving'. Together they form a unique fingerprint.

Cite this