Deviation from Perfect Performance Measures the Diagnostic Utility of Eyewitness Lineups but Partial Area Under the ROC Curve Does Not

Andrew M. Smith, James Michael Lampinen, Gary L. Wells, Laura Smalarz, Simona Mackovichova

Research output: Contribution to journalReview articlepeer-review

5 Scopus citations

Abstract

When one lineup identification procedure leads to both fewer innocent–suspect identifications and fewer culprit identifications than does some other lineup procedure, it is difficult to determine whether the procedures differ in diagnostic accuracy. In an influential article, Wixted and Mickes (2012) argued that measures of probative value do not inform diagnostic accuracy in these situations but that the partial area under the receiver operator characteristic curve (pAUC) does. In more recent research, we have found that pAUC does not necessarily indicate which of two lineup procedures has higher expected utility. When two lineup procedures produce different innocent-suspect identification rates, it leads to differential truncation of the ROC curves. As a result, diagnostic utility as measured by the pAUC is confounded with witness confidence level. We introduce a novel receiver operator characteristic measure, deviation from perfect performance (DPP), that unconfounds diagnostic utility and witness confidence level and consistently indicates which of two lineup procedures has higher expected utility. Our findings suggest that eyewitness scientists should abandon pAUC as a measure of diagnostic accuracy and embrace deviation from perfect performance.

Original languageEnglish (US)
Pages (from-to)50-59
Number of pages10
JournalJournal of Applied Research in Memory and Cognition
Volume8
Issue number1
DOIs
StatePublished - Mar 2019

Keywords

  • Area under the curve
  • Deviation from perfect performance
  • Diagnostic accuracy
  • Eyewitness science
  • ROC analysis

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Clinical Psychology
  • Applied Psychology

Fingerprint Dive into the research topics of 'Deviation from Perfect Performance Measures the Diagnostic Utility of Eyewitness Lineups but Partial Area Under the ROC Curve Does Not'. Together they form a unique fingerprint.

Cite this