Deviation from Perfect Performance Measures the Diagnostic Utility of Eyewitness Lineups but Partial Area Under the ROC Curve Does Not

When one lineup identification procedure leads to both fewer innocent–suspect identifications and fewer culprit identifications than does some other lineup procedure, it is difficult to determine whether the procedures differ in diagnostic accuracy. In an influential article, Wixted and Mickes (2012) argued that measures of probative value do not inform diagnostic accuracy in these situations but that the partial area under the receiver operator characteristic curve (pAUC) does. In more recent research, we have found that pAUC does not necessarily indicate which of two lineup procedures has higher expected utility. When two lineup procedures produce different innocent-suspect identification rates, it leads to differential truncation of the ROC curves. As a result, diagnostic utility as measured by the pAUC is confounded with witness confidence level. We introduce a novel receiver operator characteristic measure, deviation from perfect performance (DPP), that unconfounds diagnostic utility and witness confidence level and consistently indicates which of two lineup procedures has higher expected utility. Our findings suggest that eyewitness scientists should abandon pAUC as a measure of diagnostic accuracy and embrace deviation from perfect performance.