Validity of a computational linguistics-derived automated health literacy measure across race/ethnicity: Findings from the eclippse project

Dean Schillinger, Renu Balyan, Scott Crossley, Danielle McNamara, Andrew Karter

Research output: Contribution to journalArticlepeer-review

Abstract

Limited health literacy (HL) partially mediates health disparities. Measurement constraints, including lack of validity assessment across racial/ethnic groups and administration challenges, have undermined the field and impeded scaling of HL interventions. We employed computational linguistics to develop an automated and novel HL measure, analyzing >300,000 messages sent by >9,000 diabetes patients via a patient portal to create a Literacy Profiles. We carried out stratified analyses among White/non-Hispanics, Black/ non-Hispanics, Hispanics, and Asian/Pacific Islanders to determine if the Literacy Profile has comparable criterion and predictive validities. We discovered that criterion validity was consistently high across all groups (c-statistics 0.82–0.89). We observed consistent relation-ships across racial/ethnic groups between HL and outcomes, including communication, adherence, hypoglycemia, diabetes control, and ED utilization. While concerns have arisen regarding bias in AI, the automated Literacy Profile appears sufficiently valid across race/ ethnicity, enabling HL measurement at a scale that could improve clinical care and population health among diverse populations.

Original languageEnglish (US)
Pages (from-to)347-365
Number of pages19
JournalJournal of health care for the poor and underserved
Volume32
Issue number2
DOIs
StatePublished - 2021
Externally publishedYes

Keywords

  • Artificial intelligence
  • Communication
  • Computational linguistics
  • Diabetes
  • Health disparities
  • Health literacy
  • Machine learning
  • Validation study

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health

Fingerprint

Dive into the research topics of 'Validity of a computational linguistics-derived automated health literacy measure across race/ethnicity: Findings from the eclippse project'. Together they form a unique fingerprint.

Cite this