Evaluating the Quality of Health Information in a Changing Digital Ecosystem

Alla Keselman, Catherine Arnott Smith, Anita C. Murcko, David Kaufman

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

BACKGROUND: Critical evaluation of online health information has always been central to consumer health informatics. However, with the emergence of new Web media platforms and the ubiquity of social media, the issue has taken on a new dimension and urgency. At the same time, many established existing information quality evaluation guidelines address information characteristics other than the content (eg, authority and currency), target information creators rather than users as their main audience, or do not address information presented via novel Web technologies. OBJECTIVE: The aim of this formative study was to (1) develop a methodological approach for analyzing health-related Web pages and (2) apply it to a set of relevant Web pages. METHODS: This qualitative study analyzed 25 type 2 diabetes pages, which were derived from the results of a Google search with the keywords "diabetes," "reversal," and "natural." The coding scheme, developed via a combination of theory- and data-driven approaches, includes 5 categories from existing guidelines (resource type, information authority, validity of background information sources, objectivity, and currency) and 7 novel categories (treatment or reversal method, promises and certainty, criticisms of establishment, emotional appeal, vocabulary, rhetoric and presentation, and use of science in argumentation). The coding involves both categorical judgment and in-depth narrative characterization. On establishing satisfactory level of agreement on the narrative coding, the team coded the complete dataset of 25 pages. RESULTS: The results set included "traditional" static pages, videos, and digitized versions of printed newspapers or magazine articles. Treatments proposed by the pages included a mixture of conventional evidence-based treatments (eg, healthy balanced diet exercise) and unconventional treatments (eg, dietary supplements, optimizing gut flora). Most pages either promised or strongly implied high likelihood of complete recovery. Pages varied greatly with respect to the authors' stated background and credentials as well as the information sources they referenced or mentioned. The majority included criticisms of the traditional health care establishment. Many sold commercial products ranging from dietary supplements to books. The pages frequently used colloquial language. A significant number included emotional personal anecdotes, made positive mentions of the word cure, and included references to nature as a positive healing force. Most pages presented some biological explanations of their proposed treatments. Some of the explanations involved the level of complexity well beyond the level of an educated layperson. CONCLUSIONS: Both traditional and data-driven categories of codes used in this work yielded insights about the resources and highlighted challenges faced by their users. This exploratory study underscores the challenges of consumer health information seeking and the importance of developing support tools that would help users seek, evaluate, and analyze information in the changing digital ecosystem.

Original languageEnglish (US)
Pages (from-to)e11129
JournalJournal of medical Internet research
Volume21
Issue number2
DOIs
StatePublished - Feb 8 2019

Fingerprint

Ecosystem
Dietary Supplements
Health
Consumer Health Information
Anecdotes
Guidelines
Social Media
Newspapers
Informatics
Vocabulary
Type 2 Diabetes Mellitus
Language
Technology
Delivery of Health Care
Healthy Diet
Gastrointestinal Microbiome
Datasets

Keywords

  • consumer health information
  • eHealth
  • eHealth literacy
  • health literacy
  • information evaluation
  • information literacy
  • information quality
  • type 2 diabetes mellitus

ASJC Scopus subject areas

  • Health Informatics

Cite this

Evaluating the Quality of Health Information in a Changing Digital Ecosystem. / Keselman, Alla; Arnott Smith, Catherine; Murcko, Anita C.; Kaufman, David.

In: Journal of medical Internet research, Vol. 21, No. 2, 08.02.2019, p. e11129.

Research output: Contribution to journalArticle

Keselman, Alla ; Arnott Smith, Catherine ; Murcko, Anita C. ; Kaufman, David. / Evaluating the Quality of Health Information in a Changing Digital Ecosystem. In: Journal of medical Internet research. 2019 ; Vol. 21, No. 2. pp. e11129.
@article{00ecee6e13704a24806d3c0c90e3199b,
title = "Evaluating the Quality of Health Information in a Changing Digital Ecosystem",
abstract = "BACKGROUND: Critical evaluation of online health information has always been central to consumer health informatics. However, with the emergence of new Web media platforms and the ubiquity of social media, the issue has taken on a new dimension and urgency. At the same time, many established existing information quality evaluation guidelines address information characteristics other than the content (eg, authority and currency), target information creators rather than users as their main audience, or do not address information presented via novel Web technologies. OBJECTIVE: The aim of this formative study was to (1) develop a methodological approach for analyzing health-related Web pages and (2) apply it to a set of relevant Web pages. METHODS: This qualitative study analyzed 25 type 2 diabetes pages, which were derived from the results of a Google search with the keywords {"}diabetes,{"} {"}reversal,{"} and {"}natural.{"} The coding scheme, developed via a combination of theory- and data-driven approaches, includes 5 categories from existing guidelines (resource type, information authority, validity of background information sources, objectivity, and currency) and 7 novel categories (treatment or reversal method, promises and certainty, criticisms of establishment, emotional appeal, vocabulary, rhetoric and presentation, and use of science in argumentation). The coding involves both categorical judgment and in-depth narrative characterization. On establishing satisfactory level of agreement on the narrative coding, the team coded the complete dataset of 25 pages. RESULTS: The results set included {"}traditional{"} static pages, videos, and digitized versions of printed newspapers or magazine articles. Treatments proposed by the pages included a mixture of conventional evidence-based treatments (eg, healthy balanced diet exercise) and unconventional treatments (eg, dietary supplements, optimizing gut flora). Most pages either promised or strongly implied high likelihood of complete recovery. Pages varied greatly with respect to the authors' stated background and credentials as well as the information sources they referenced or mentioned. The majority included criticisms of the traditional health care establishment. Many sold commercial products ranging from dietary supplements to books. The pages frequently used colloquial language. A significant number included emotional personal anecdotes, made positive mentions of the word cure, and included references to nature as a positive healing force. Most pages presented some biological explanations of their proposed treatments. Some of the explanations involved the level of complexity well beyond the level of an educated layperson. CONCLUSIONS: Both traditional and data-driven categories of codes used in this work yielded insights about the resources and highlighted challenges faced by their users. This exploratory study underscores the challenges of consumer health information seeking and the importance of developing support tools that would help users seek, evaluate, and analyze information in the changing digital ecosystem.",
keywords = "consumer health information, eHealth, eHealth literacy, health literacy, information evaluation, information literacy, information quality, type 2 diabetes mellitus",
author = "Alla Keselman and {Arnott Smith}, Catherine and Murcko, {Anita C.} and David Kaufman",
year = "2019",
month = "2",
day = "8",
doi = "10.2196/11129",
language = "English (US)",
volume = "21",
pages = "e11129",
journal = "Journal of Medical Internet Research",
issn = "1439-4456",
publisher = "Journal of medical Internet Research",
number = "2",

}

TY - JOUR

T1 - Evaluating the Quality of Health Information in a Changing Digital Ecosystem

AU - Keselman, Alla

AU - Arnott Smith, Catherine

AU - Murcko, Anita C.

AU - Kaufman, David

PY - 2019/2/8

Y1 - 2019/2/8

N2 - BACKGROUND: Critical evaluation of online health information has always been central to consumer health informatics. However, with the emergence of new Web media platforms and the ubiquity of social media, the issue has taken on a new dimension and urgency. At the same time, many established existing information quality evaluation guidelines address information characteristics other than the content (eg, authority and currency), target information creators rather than users as their main audience, or do not address information presented via novel Web technologies. OBJECTIVE: The aim of this formative study was to (1) develop a methodological approach for analyzing health-related Web pages and (2) apply it to a set of relevant Web pages. METHODS: This qualitative study analyzed 25 type 2 diabetes pages, which were derived from the results of a Google search with the keywords "diabetes," "reversal," and "natural." The coding scheme, developed via a combination of theory- and data-driven approaches, includes 5 categories from existing guidelines (resource type, information authority, validity of background information sources, objectivity, and currency) and 7 novel categories (treatment or reversal method, promises and certainty, criticisms of establishment, emotional appeal, vocabulary, rhetoric and presentation, and use of science in argumentation). The coding involves both categorical judgment and in-depth narrative characterization. On establishing satisfactory level of agreement on the narrative coding, the team coded the complete dataset of 25 pages. RESULTS: The results set included "traditional" static pages, videos, and digitized versions of printed newspapers or magazine articles. Treatments proposed by the pages included a mixture of conventional evidence-based treatments (eg, healthy balanced diet exercise) and unconventional treatments (eg, dietary supplements, optimizing gut flora). Most pages either promised or strongly implied high likelihood of complete recovery. Pages varied greatly with respect to the authors' stated background and credentials as well as the information sources they referenced or mentioned. The majority included criticisms of the traditional health care establishment. Many sold commercial products ranging from dietary supplements to books. The pages frequently used colloquial language. A significant number included emotional personal anecdotes, made positive mentions of the word cure, and included references to nature as a positive healing force. Most pages presented some biological explanations of their proposed treatments. Some of the explanations involved the level of complexity well beyond the level of an educated layperson. CONCLUSIONS: Both traditional and data-driven categories of codes used in this work yielded insights about the resources and highlighted challenges faced by their users. This exploratory study underscores the challenges of consumer health information seeking and the importance of developing support tools that would help users seek, evaluate, and analyze information in the changing digital ecosystem.

AB - BACKGROUND: Critical evaluation of online health information has always been central to consumer health informatics. However, with the emergence of new Web media platforms and the ubiquity of social media, the issue has taken on a new dimension and urgency. At the same time, many established existing information quality evaluation guidelines address information characteristics other than the content (eg, authority and currency), target information creators rather than users as their main audience, or do not address information presented via novel Web technologies. OBJECTIVE: The aim of this formative study was to (1) develop a methodological approach for analyzing health-related Web pages and (2) apply it to a set of relevant Web pages. METHODS: This qualitative study analyzed 25 type 2 diabetes pages, which were derived from the results of a Google search with the keywords "diabetes," "reversal," and "natural." The coding scheme, developed via a combination of theory- and data-driven approaches, includes 5 categories from existing guidelines (resource type, information authority, validity of background information sources, objectivity, and currency) and 7 novel categories (treatment or reversal method, promises and certainty, criticisms of establishment, emotional appeal, vocabulary, rhetoric and presentation, and use of science in argumentation). The coding involves both categorical judgment and in-depth narrative characterization. On establishing satisfactory level of agreement on the narrative coding, the team coded the complete dataset of 25 pages. RESULTS: The results set included "traditional" static pages, videos, and digitized versions of printed newspapers or magazine articles. Treatments proposed by the pages included a mixture of conventional evidence-based treatments (eg, healthy balanced diet exercise) and unconventional treatments (eg, dietary supplements, optimizing gut flora). Most pages either promised or strongly implied high likelihood of complete recovery. Pages varied greatly with respect to the authors' stated background and credentials as well as the information sources they referenced or mentioned. The majority included criticisms of the traditional health care establishment. Many sold commercial products ranging from dietary supplements to books. The pages frequently used colloquial language. A significant number included emotional personal anecdotes, made positive mentions of the word cure, and included references to nature as a positive healing force. Most pages presented some biological explanations of their proposed treatments. Some of the explanations involved the level of complexity well beyond the level of an educated layperson. CONCLUSIONS: Both traditional and data-driven categories of codes used in this work yielded insights about the resources and highlighted challenges faced by their users. This exploratory study underscores the challenges of consumer health information seeking and the importance of developing support tools that would help users seek, evaluate, and analyze information in the changing digital ecosystem.

KW - consumer health information

KW - eHealth

KW - eHealth literacy

KW - health literacy

KW - information evaluation

KW - information literacy

KW - information quality

KW - type 2 diabetes mellitus

UR - http://www.scopus.com/inward/record.url?scp=85061262516&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061262516&partnerID=8YFLogxK

U2 - 10.2196/11129

DO - 10.2196/11129

M3 - Article

VL - 21

SP - e11129

JO - Journal of Medical Internet Research

JF - Journal of Medical Internet Research

SN - 1439-4456

IS - 2

ER -