High accuracy detection of collaboration from log data and superficial speech features

Sree Aurovindh Viswanathan, Kurt Vanlehn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Effective collaborative behavior between students is neither spontaneous nor continuous. A system that can measure collaboration in real-time may be useful. For instance, it could alert an instructor that a group needs attention. We tested whether superficial measures of speech and user interactions of students would suffice for measuring collaboration. As pairs of students solved complex math problems on tablets, their speech and tablet gestures were recorded. These data and multi-camera videos were used by humans to code episodes as collaborative vs. various kinds of non-collaboration. Using just the speech and tablet log data, several detectors were machine learned. The best had an overall accuracy of 96% (Kappa=0.92), which is higher than earlier attempts to use speech and log data for detecting collaboration. The improved accuracy appears to be due both to our analytic methods and to the particular mathematical task, which involves moving objects.

Original languageEnglish (US)
Title of host publicationMaking a Difference
Subtitle of host publicationPrioritizing Equity and Access in CSCL - 12th International Conference on Computer Supported Collaborative Learning, CSCL 2017 - Conference Proceedings
EditorsBrian K. Smith, Marcela Borge, Emma Mercier, Kyu Yon Lim
PublisherInternational Society of the Learning Sciences (ISLS)
Pages335-342
Number of pages8
ISBN (Electronic)9780990355007
StatePublished - Jan 1 2017
Externally publishedYes
Event12th International Conference on Computer Supported Collaborative Learning - Making a Difference: Prioritizing Equity and Access in CSCL, CSCL 2017 - Philadelphia, United States
Duration: Jun 18 2017Jun 22 2017

Publication series

NameComputer-Supported Collaborative Learning Conference, CSCL
Volume1
ISSN (Print)1573-4552

Conference

Conference12th International Conference on Computer Supported Collaborative Learning - Making a Difference: Prioritizing Equity and Access in CSCL, CSCL 2017
CountryUnited States
CityPhiladelphia
Period6/18/176/22/17

Fingerprint

Students
student
Video cameras
instructor
video
Detectors
interaction
Group
time

Keywords

  • Collaboration detection
  • Cooperation detection
  • Learning analytics

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Education

Cite this

Viswanathan, S. A., & Vanlehn, K. (2017). High accuracy detection of collaboration from log data and superficial speech features. In B. K. Smith, M. Borge, E. Mercier, & K. Y. Lim (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL - 12th International Conference on Computer Supported Collaborative Learning, CSCL 2017 - Conference Proceedings (pp. 335-342). (Computer-Supported Collaborative Learning Conference, CSCL; Vol. 1). International Society of the Learning Sciences (ISLS).

High accuracy detection of collaboration from log data and superficial speech features. / Viswanathan, Sree Aurovindh; Vanlehn, Kurt.

Making a Difference: Prioritizing Equity and Access in CSCL - 12th International Conference on Computer Supported Collaborative Learning, CSCL 2017 - Conference Proceedings. ed. / Brian K. Smith; Marcela Borge; Emma Mercier; Kyu Yon Lim. International Society of the Learning Sciences (ISLS), 2017. p. 335-342 (Computer-Supported Collaborative Learning Conference, CSCL; Vol. 1).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Viswanathan, SA & Vanlehn, K 2017, High accuracy detection of collaboration from log data and superficial speech features. in BK Smith, M Borge, E Mercier & KY Lim (eds), Making a Difference: Prioritizing Equity and Access in CSCL - 12th International Conference on Computer Supported Collaborative Learning, CSCL 2017 - Conference Proceedings. Computer-Supported Collaborative Learning Conference, CSCL, vol. 1, International Society of the Learning Sciences (ISLS), pp. 335-342, 12th International Conference on Computer Supported Collaborative Learning - Making a Difference: Prioritizing Equity and Access in CSCL, CSCL 2017, Philadelphia, United States, 6/18/17.
Viswanathan SA, Vanlehn K. High accuracy detection of collaboration from log data and superficial speech features. In Smith BK, Borge M, Mercier E, Lim KY, editors, Making a Difference: Prioritizing Equity and Access in CSCL - 12th International Conference on Computer Supported Collaborative Learning, CSCL 2017 - Conference Proceedings. International Society of the Learning Sciences (ISLS). 2017. p. 335-342. (Computer-Supported Collaborative Learning Conference, CSCL).
Viswanathan, Sree Aurovindh ; Vanlehn, Kurt. / High accuracy detection of collaboration from log data and superficial speech features. Making a Difference: Prioritizing Equity and Access in CSCL - 12th International Conference on Computer Supported Collaborative Learning, CSCL 2017 - Conference Proceedings. editor / Brian K. Smith ; Marcela Borge ; Emma Mercier ; Kyu Yon Lim. International Society of the Learning Sciences (ISLS), 2017. pp. 335-342 (Computer-Supported Collaborative Learning Conference, CSCL).
@inproceedings{36100ced8ba74dffad841a3670e7734c,
title = "High accuracy detection of collaboration from log data and superficial speech features",
abstract = "Effective collaborative behavior between students is neither spontaneous nor continuous. A system that can measure collaboration in real-time may be useful. For instance, it could alert an instructor that a group needs attention. We tested whether superficial measures of speech and user interactions of students would suffice for measuring collaboration. As pairs of students solved complex math problems on tablets, their speech and tablet gestures were recorded. These data and multi-camera videos were used by humans to code episodes as collaborative vs. various kinds of non-collaboration. Using just the speech and tablet log data, several detectors were machine learned. The best had an overall accuracy of 96{\%} (Kappa=0.92), which is higher than earlier attempts to use speech and log data for detecting collaboration. The improved accuracy appears to be due both to our analytic methods and to the particular mathematical task, which involves moving objects.",
keywords = "Collaboration detection, Cooperation detection, Learning analytics",
author = "Viswanathan, {Sree Aurovindh} and Kurt Vanlehn",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
series = "Computer-Supported Collaborative Learning Conference, CSCL",
publisher = "International Society of the Learning Sciences (ISLS)",
pages = "335--342",
editor = "Smith, {Brian K.} and Marcela Borge and Emma Mercier and Lim, {Kyu Yon}",
booktitle = "Making a Difference",

}

TY - GEN

T1 - High accuracy detection of collaboration from log data and superficial speech features

AU - Viswanathan, Sree Aurovindh

AU - Vanlehn, Kurt

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Effective collaborative behavior between students is neither spontaneous nor continuous. A system that can measure collaboration in real-time may be useful. For instance, it could alert an instructor that a group needs attention. We tested whether superficial measures of speech and user interactions of students would suffice for measuring collaboration. As pairs of students solved complex math problems on tablets, their speech and tablet gestures were recorded. These data and multi-camera videos were used by humans to code episodes as collaborative vs. various kinds of non-collaboration. Using just the speech and tablet log data, several detectors were machine learned. The best had an overall accuracy of 96% (Kappa=0.92), which is higher than earlier attempts to use speech and log data for detecting collaboration. The improved accuracy appears to be due both to our analytic methods and to the particular mathematical task, which involves moving objects.

AB - Effective collaborative behavior between students is neither spontaneous nor continuous. A system that can measure collaboration in real-time may be useful. For instance, it could alert an instructor that a group needs attention. We tested whether superficial measures of speech and user interactions of students would suffice for measuring collaboration. As pairs of students solved complex math problems on tablets, their speech and tablet gestures were recorded. These data and multi-camera videos were used by humans to code episodes as collaborative vs. various kinds of non-collaboration. Using just the speech and tablet log data, several detectors were machine learned. The best had an overall accuracy of 96% (Kappa=0.92), which is higher than earlier attempts to use speech and log data for detecting collaboration. The improved accuracy appears to be due both to our analytic methods and to the particular mathematical task, which involves moving objects.

KW - Collaboration detection

KW - Cooperation detection

KW - Learning analytics

UR - http://www.scopus.com/inward/record.url?scp=85073390650&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073390650&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85073390650

T3 - Computer-Supported Collaborative Learning Conference, CSCL

SP - 335

EP - 342

BT - Making a Difference

A2 - Smith, Brian K.

A2 - Borge, Marcela

A2 - Mercier, Emma

A2 - Lim, Kyu Yon

PB - International Society of the Learning Sciences (ISLS)

ER -