ROC analysis of lineups does not measure underlying discriminability and has limited value

Gary L. Wells, Laura Smalarz, Andrew M. Smith

Research output: Contribution to journalArticlepeer-review

40 Scopus citations

Abstract

Some researchers have been arguing that eyewitness identification data from lineups should be analyzed using Receiver Operating Characteristic (ROC) analysis because it purportedly measures underlying discriminability. But ROC analysis, which was designed for 2. ×. 2 tasks, does not fit the 3. ×. 2 structure of lineups. Accordingly, ROC proponents force lineup data into a 2. ×. 2 structure by treating false-positive identifications of lineup fillers as though they were rejections. Using data from lineups versus showups, we illustrate how this approach misfires as a measure of underlying discriminability. Moreover, treating false-positive identifications of fillers as if they were rejections hides one of the most important phenomena in eyewitness lineups, namely filler siphoning. Filler siphoning reduces the risk of mistaken identification by drawing false-positive identifications away from the innocent suspect and onto lineup fillers. We show that ROC analysis confuses filler siphoning with an improvement in underlying discriminability, thereby fostering misleading theoretical conclusions about how lineups work.

Original languageEnglish (US)
Pages (from-to)313-317
Number of pages5
JournalJournal of Applied Research in Memory and Cognition
Volume4
Issue number4
DOIs
StatePublished - Dec 1 2015
Externally publishedYes

Keywords

  • Bayesian analysis
  • Eyewitness identification
  • Lineup discriminability
  • Lineup fillers
  • Lineups
  • Receiver Operating Characteristic (ROC) analysis
  • Showups

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Clinical Psychology
  • Applied Psychology

Fingerprint

Dive into the research topics of 'ROC analysis of lineups does not measure underlying discriminability and has limited value'. Together they form a unique fingerprint.

Cite this