Coding Qualitative Data at Scale: Guidance for Large Coder Teams Based on 18 Studies

Melissa Beresford, Amber Wutich, Margaret V. du Bray, Alissa Ruth, Rhian Stotts, Cindi SturtzSreetharan, Alexandra Brewis

Research output: Contribution to journalArticlepeer-review

Abstract

We outline a process for using large coder teams (10 + coders) to code large-scale qualitative data sets. The process reflects experience recruiting and managing large teams of novice and trainee coders for 18 projects in the last decade, each engaging a coding team of 12 (minimum) to 54 (maximum) coders. We identify four unique challenges to large coder teams that are not presently discussed in the methodological literature: (1) recruiting and training coders, (2) providing coder compensation and incentives, (3) maintaining data quality and ensuring coding reliability at scale, and (4) building team cohesion and morale. For each challenge, we provide associated guidance. We conclude with a discussion of advantages and disadvantages of large coder teams for qualitative research and provide notes of caution for anyone considering hiring and/or managing large coder teams for research (whether in academia, government and non-profit sectors, or industry).

Original languageEnglish (US)
JournalInternational Journal of Qualitative Methods
Volume21
DOIs
StatePublished - Jan 11 2022

Keywords

  • collaborative research
  • intercoder agreement
  • intercoder reliability
  • qualitative coding
  • qualitative data analysis
  • team-based coding
  • text analysis

ASJC Scopus subject areas

  • Education

Fingerprint

Dive into the research topics of 'Coding Qualitative Data at Scale: Guidance for Large Coder Teams Based on 18 Studies'. Together they form a unique fingerprint.

Cite this