BayesRank: A bayesian approach to ranked peer grading

Andrew E. Waters, David Tinapple, Richard G. Baraniuk

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations

Abstract

Advances in online and computer supported education afford exciting opportunities to revolutionize the classroom, while also presenting a number of new challenges not faced in traditional educational settings. Foremost among these challenges is the problem of accurately and efficiently evaluating learner work as the class size grows, which is directly related to the larger goal of providing quality, timely, and actionable formative feedback. Recently there has been a surge in interest in using peer grading methods coupled with machine learning to accurately and fairly evaluate learner work while alleviating the instructor bottleneck and grading overload. Prior work in peer grading almost exclusively focuses on numerically scored grades - either real-valued or ordinal. In this work, we consider the implications of peer ranking in which learners rank a small subset of peer work from strongest to weakest, and propose new types of computational analyses that can be applied to this ranking data. We adopt a Bayesian approach to the ranked peer grading problem and develop a novel model and method for utilizing ranked peer-grading data. We additionally develop a novel procedure for adaptively identifying which work should be ranked by particular peers in order to dynamically resolve ambiguity in the data and rapidly resolve a clearer picture of learner performance. We showcase our results on both synthetic and several real-world educational datasets.

Original languageEnglish (US)
Title of host publicationL@S 2015 - 2nd ACM Conference on Learning at Scale
PublisherAssociation for Computing Machinery, Inc
Pages177-183
Number of pages7
ISBN (Electronic)9781450334112
DOIs
StatePublished - Mar 14 2015
Event2nd ACM Conference on Learning at Scale, L@S 2015 - Vancouver, Canada
Duration: Mar 14 2015Mar 18 2015

Publication series

NameL@S 2015 - 2nd ACM Conference on Learning at Scale

Other

Other2nd ACM Conference on Learning at Scale, L@S 2015
CountryCanada
CityVancouver
Period3/14/153/18/15

ASJC Scopus subject areas

  • Software
  • Education
  • Computer Science Applications
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'BayesRank: A bayesian approach to ranked peer grading'. Together they form a unique fingerprint.

  • Cite this

    Waters, A. E., Tinapple, D., & Baraniuk, R. G. (2015). BayesRank: A bayesian approach to ranked peer grading. In L@S 2015 - 2nd ACM Conference on Learning at Scale (pp. 177-183). (L@S 2015 - 2nd ACM Conference on Learning at Scale). Association for Computing Machinery, Inc. https://doi.org/10.1145/2724660.2724672