A Bayesian Item Response Theory Approach to Symptom Validity Detection: Evaluating Psychological Screening Inventory-2 Response Profile Likelihoods

Michael L. Thomas, Richard I. Lanyon

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Invalid data in forensic assessment are most often indicated by excessive endorsement of psychiatric symptoms. Although this method of identifying invalid profiles is generally effective, it does not make use of all conditional dependencies in data. Modern psychometric methodologies can be used to identify aberrant response profiles through model-based indices known as person fit statistics. Specifically, the likelihoods of examinees' response profiles can be compared against observed or simulated likelihoods that are derived from empirical models of emotional and psychiatric functioning. This study demonstrates how person fit indices based on item response theory models can be used to detect misfitting response profiles in forensic assessment. Archival data from the Psychological Screening Inventory-2 (R. I. Lanyon, 2010a) were evaluated with Bayesian estimation and posterior predictive model checking to compare the response profile log-likelihoods of 74 forensic participants with 1,046 normative participants. Results suggest 61 % of forensic examinees but only 5 % of normative examinees had misfitting data. Misfitting "fake bad" forensic profiles appeared to be associated with overly discrepant endorsement of symptoms, and misfitting "fake good" forensic profiles appeared to be associated with overly narrow endorsement of symptoms. The high rate of misfit among forensic examinees challenges the appropriateness of basing interpretations of forensic data on reliability and validity coefficients from normative samples. However, because aspects of the methodology are still untested in forensic and clinical assessment (e. g., the use of priors in this study), future research is needed to evaluate its appropriateness for clinical practice.

Original languageEnglish (US)
Pages (from-to)221-234
Number of pages14
JournalPsychological Injury and Law
Volume5
Issue number3-4
DOIs
StatePublished - Dec 2012

Fingerprint

Psychiatry
Psychology
Equipment and Supplies
Psychometrics
Reproducibility of Results
human being
model theory
methodology
predictive model
psychometrics
statistics
detection
screening
interpretation
Dependency (Psychology)
index

Keywords

  • Item response theory
  • Person fit
  • Psychological screening inventory-2
  • Symptom validity

ASJC Scopus subject areas

  • Ecology
  • Psychiatry and Mental health
  • Law

Cite this

A Bayesian Item Response Theory Approach to Symptom Validity Detection : Evaluating Psychological Screening Inventory-2 Response Profile Likelihoods. / Thomas, Michael L.; Lanyon, Richard I.

In: Psychological Injury and Law, Vol. 5, No. 3-4, 12.2012, p. 221-234.

Research output: Contribution to journalArticle

@article{5def0cafeca845f8a5a03b4d557f3538,
title = "A Bayesian Item Response Theory Approach to Symptom Validity Detection: Evaluating Psychological Screening Inventory-2 Response Profile Likelihoods",
abstract = "Invalid data in forensic assessment are most often indicated by excessive endorsement of psychiatric symptoms. Although this method of identifying invalid profiles is generally effective, it does not make use of all conditional dependencies in data. Modern psychometric methodologies can be used to identify aberrant response profiles through model-based indices known as person fit statistics. Specifically, the likelihoods of examinees' response profiles can be compared against observed or simulated likelihoods that are derived from empirical models of emotional and psychiatric functioning. This study demonstrates how person fit indices based on item response theory models can be used to detect misfitting response profiles in forensic assessment. Archival data from the Psychological Screening Inventory-2 (R. I. Lanyon, 2010a) were evaluated with Bayesian estimation and posterior predictive model checking to compare the response profile log-likelihoods of 74 forensic participants with 1,046 normative participants. Results suggest 61 {\%} of forensic examinees but only 5 {\%} of normative examinees had misfitting data. Misfitting {"}fake bad{"} forensic profiles appeared to be associated with overly discrepant endorsement of symptoms, and misfitting {"}fake good{"} forensic profiles appeared to be associated with overly narrow endorsement of symptoms. The high rate of misfit among forensic examinees challenges the appropriateness of basing interpretations of forensic data on reliability and validity coefficients from normative samples. However, because aspects of the methodology are still untested in forensic and clinical assessment (e. g., the use of priors in this study), future research is needed to evaluate its appropriateness for clinical practice.",
keywords = "Item response theory, Person fit, Psychological screening inventory-2, Symptom validity",
author = "Thomas, {Michael L.} and Lanyon, {Richard I.}",
year = "2012",
month = "12",
doi = "10.1007/s12207-012-9129-4",
language = "English (US)",
volume = "5",
pages = "221--234",
journal = "Psychological Injury and Law",
issn = "1938-971X",
publisher = "Springer New York",
number = "3-4",

}

TY - JOUR

T1 - A Bayesian Item Response Theory Approach to Symptom Validity Detection

T2 - Evaluating Psychological Screening Inventory-2 Response Profile Likelihoods

AU - Thomas, Michael L.

AU - Lanyon, Richard I.

PY - 2012/12

Y1 - 2012/12

N2 - Invalid data in forensic assessment are most often indicated by excessive endorsement of psychiatric symptoms. Although this method of identifying invalid profiles is generally effective, it does not make use of all conditional dependencies in data. Modern psychometric methodologies can be used to identify aberrant response profiles through model-based indices known as person fit statistics. Specifically, the likelihoods of examinees' response profiles can be compared against observed or simulated likelihoods that are derived from empirical models of emotional and psychiatric functioning. This study demonstrates how person fit indices based on item response theory models can be used to detect misfitting response profiles in forensic assessment. Archival data from the Psychological Screening Inventory-2 (R. I. Lanyon, 2010a) were evaluated with Bayesian estimation and posterior predictive model checking to compare the response profile log-likelihoods of 74 forensic participants with 1,046 normative participants. Results suggest 61 % of forensic examinees but only 5 % of normative examinees had misfitting data. Misfitting "fake bad" forensic profiles appeared to be associated with overly discrepant endorsement of symptoms, and misfitting "fake good" forensic profiles appeared to be associated with overly narrow endorsement of symptoms. The high rate of misfit among forensic examinees challenges the appropriateness of basing interpretations of forensic data on reliability and validity coefficients from normative samples. However, because aspects of the methodology are still untested in forensic and clinical assessment (e. g., the use of priors in this study), future research is needed to evaluate its appropriateness for clinical practice.

AB - Invalid data in forensic assessment are most often indicated by excessive endorsement of psychiatric symptoms. Although this method of identifying invalid profiles is generally effective, it does not make use of all conditional dependencies in data. Modern psychometric methodologies can be used to identify aberrant response profiles through model-based indices known as person fit statistics. Specifically, the likelihoods of examinees' response profiles can be compared against observed or simulated likelihoods that are derived from empirical models of emotional and psychiatric functioning. This study demonstrates how person fit indices based on item response theory models can be used to detect misfitting response profiles in forensic assessment. Archival data from the Psychological Screening Inventory-2 (R. I. Lanyon, 2010a) were evaluated with Bayesian estimation and posterior predictive model checking to compare the response profile log-likelihoods of 74 forensic participants with 1,046 normative participants. Results suggest 61 % of forensic examinees but only 5 % of normative examinees had misfitting data. Misfitting "fake bad" forensic profiles appeared to be associated with overly discrepant endorsement of symptoms, and misfitting "fake good" forensic profiles appeared to be associated with overly narrow endorsement of symptoms. The high rate of misfit among forensic examinees challenges the appropriateness of basing interpretations of forensic data on reliability and validity coefficients from normative samples. However, because aspects of the methodology are still untested in forensic and clinical assessment (e. g., the use of priors in this study), future research is needed to evaluate its appropriateness for clinical practice.

KW - Item response theory

KW - Person fit

KW - Psychological screening inventory-2

KW - Symptom validity

UR - http://www.scopus.com/inward/record.url?scp=84871291888&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84871291888&partnerID=8YFLogxK

U2 - 10.1007/s12207-012-9129-4

DO - 10.1007/s12207-012-9129-4

M3 - Article

AN - SCOPUS:84871291888

VL - 5

SP - 221

EP - 234

JO - Psychological Injury and Law

JF - Psychological Injury and Law

SN - 1938-971X

IS - 3-4

ER -