TY - GEN
T1 - Modeling team-level multimodal dynamics during multiparty collaboration
AU - Eloy, Lucca
AU - Stewart, Angela E.B.
AU - Amon, Mary J.
AU - Reindhardt, Caroline
AU - Michaels, Amanda
AU - Sun, Chen
AU - Shute, Valerie
AU - Duran, Nicholas D.
AU - D'Mello, Sidney K.
N1 - Funding Information:
This research was supported by the National Science Foundation (NSF DUE 1745442) and the Institute of Educational Sciences (IES R305A170432). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding agencies.
Publisher Copyright:
© 2019 Association for Computing Machinery.
PY - 2019/10/14
Y1 - 2019/10/14
N2 - We adopt a multimodal approach to investigating team interactions in the context of remote collaborative problem solving (CPS). Our goal is to understand multimodal patterns that emerge and their relation with collaborative outcomes. We measured speech rate, body movement, and galvanic skin response from 101 triads (303 participants) who used video conferencing software to collaboratively solve challenging levels in an educational physics game. We use multi-dimensional recurrence quantification analysis (MdRQA) to quantify patterns of team-level regularity, or repeated patterns of activity in these three modalities. We found that teams exhibit significant regularity above chance baselines. Regularity was unaffected by task factors. but had a quadratic relationship with session time in that it initially increased but then decreased as the session progressed. Importantly, teams that produce more varied behavioral patterns (irregularity) reported higher emotional valence and performed better on a subset of the problem solving tasks. Regularity did not predict arousal or subjective perceptions of the collaboration. We discuss implications of our findings for the design of systems that aim to improve collaborative outcomes by monitoring the ongoing collaboration and intervening accordingly.
AB - We adopt a multimodal approach to investigating team interactions in the context of remote collaborative problem solving (CPS). Our goal is to understand multimodal patterns that emerge and their relation with collaborative outcomes. We measured speech rate, body movement, and galvanic skin response from 101 triads (303 participants) who used video conferencing software to collaboratively solve challenging levels in an educational physics game. We use multi-dimensional recurrence quantification analysis (MdRQA) to quantify patterns of team-level regularity, or repeated patterns of activity in these three modalities. We found that teams exhibit significant regularity above chance baselines. Regularity was unaffected by task factors. but had a quadratic relationship with session time in that it initially increased but then decreased as the session progressed. Importantly, teams that produce more varied behavioral patterns (irregularity) reported higher emotional valence and performed better on a subset of the problem solving tasks. Regularity did not predict arousal or subjective perceptions of the collaboration. We discuss implications of our findings for the design of systems that aim to improve collaborative outcomes by monitoring the ongoing collaboration and intervening accordingly.
KW - Behavioral Regularity
KW - Collaborative Problem Solving
KW - MdRQA
KW - Multimodal interaction
UR - http://www.scopus.com/inward/record.url?scp=85074920870&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85074920870&partnerID=8YFLogxK
U2 - 10.1145/3340555.3353748
DO - 10.1145/3340555.3353748
M3 - Conference contribution
AN - SCOPUS:85074920870
T3 - ICMI 2019 - Proceedings of the 2019 International Conference on Multimodal Interaction
SP - 244
EP - 258
BT - ICMI 2019 - Proceedings of the 2019 International Conference on Multimodal Interaction
A2 - Gao, Wen
A2 - Ling Meng, Helen Mei
A2 - Turk, Matthew
A2 - Fussell, Susan R.
A2 - Schuller, Bjorn
A2 - Schuller, Bjorn
A2 - Song, Yale
A2 - Yu, Kai
PB - Association for Computing Machinery, Inc
T2 - 21st ACM International Conference on Multimodal Interaction, ICMI 2019
Y2 - 14 October 2019 through 18 October 2019
ER -