Evaluating the effects of cognitive support on psychiatric clinical comprehension

Venkata V. Dalai, Sana Khalid, Dinesh Gottipati, Thomas Kannampallil, Vineeth John, Brett Blatter, Vimla Patel, Trevor Cohen

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Objective: Clinicians' attention is a precious resource, which in the current healthcare practice is consumed by the cognitive demands arising from complex patient conditions, information overload, time pressure, and the need to aggregate and synthesize information from disparate sources. The ability to organize information in ways that facilitate the generation of effective diagnostic solutions is a distinguishing characteristic of expert physicians, suggesting that automated systems that organize clinical information in a similar manner may augment physicians' decision-making capabilities. In this paper, we describe the design and evaluation of a theoretically driven cognitive support system (CSS) that assists psychiatrists in their interpretation of clinical cases. The system highlights, and provides the means to navigate to, text that is organized in accordance with a set of diagnostically and therapeutically meaningful higher-level concepts. Methods and materials: To evaluate the interface, 16 psychiatry residents interpreted two clinical case scenarios, with and without the CSS. Think-aloud protocols captured during their interpretation of the cases were transcribed and analyzed qualitatively. In addition, the frequency and relative position of content related to key higher-level concepts in a verbal summary of the case were evaluated. In addition the transcripts from both groups were compared to an expert derived reference standard using latent semantic analysis (LSA). Results: Qualitative analysis showed that users of the system better attended to specific clinically important aspects of both cases when these were highlighted by the system, and revealed ways in which the system mediates hypotheses generation and evaluation. Analysis of the summary data showed differences in emphasis with and without the system. The LSA analysis suggested users of the system were more "expert-like" in their emphasis, and that cognitive support was more effective in the more complex case. Conclusions: Cognitive support impacts upon clinical comprehension. This appears to be largely helpful, but may also lead to neglect of information (such as the psychosocial history) that the system does not highlight. The results have implications for the design of CSSs for clinical narratives including the role of information organization and textual embellishments for more efficient clinical case presentation and comprehension.

Original languageEnglish (US)
Pages (from-to)91-104
Number of pages14
JournalArtificial Intelligence in Medicine
Volume62
Issue number2
DOIs
StatePublished - Oct 1 2014
Externally publishedYes

Fingerprint

Semantics
Psychiatry
Physicians
Aptitude
Decision Making
Decision making
History
Delivery of Health Care

Keywords

  • Biomedical informatics
  • Clinical comprehension
  • Cognitive science
  • Cognitive support
  • Emergency psychiatry
  • Latent semantic analysis
  • Propositional analysis
  • Psychiatry
  • Verbal protocol analysis

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Artificial Intelligence

Cite this

Dalai, V. V., Khalid, S., Gottipati, D., Kannampallil, T., John, V., Blatter, B., ... Cohen, T. (2014). Evaluating the effects of cognitive support on psychiatric clinical comprehension. Artificial Intelligence in Medicine, 62(2), 91-104. https://doi.org/10.1016/j.artmed.2014.08.002

Evaluating the effects of cognitive support on psychiatric clinical comprehension. / Dalai, Venkata V.; Khalid, Sana; Gottipati, Dinesh; Kannampallil, Thomas; John, Vineeth; Blatter, Brett; Patel, Vimla; Cohen, Trevor.

In: Artificial Intelligence in Medicine, Vol. 62, No. 2, 01.10.2014, p. 91-104.

Research output: Contribution to journalArticle

Dalai, VV, Khalid, S, Gottipati, D, Kannampallil, T, John, V, Blatter, B, Patel, V & Cohen, T 2014, 'Evaluating the effects of cognitive support on psychiatric clinical comprehension', Artificial Intelligence in Medicine, vol. 62, no. 2, pp. 91-104. https://doi.org/10.1016/j.artmed.2014.08.002
Dalai, Venkata V. ; Khalid, Sana ; Gottipati, Dinesh ; Kannampallil, Thomas ; John, Vineeth ; Blatter, Brett ; Patel, Vimla ; Cohen, Trevor. / Evaluating the effects of cognitive support on psychiatric clinical comprehension. In: Artificial Intelligence in Medicine. 2014 ; Vol. 62, No. 2. pp. 91-104.
@article{1f81f2e407e54d00a691469863005b7d,
title = "Evaluating the effects of cognitive support on psychiatric clinical comprehension",
abstract = "Objective: Clinicians' attention is a precious resource, which in the current healthcare practice is consumed by the cognitive demands arising from complex patient conditions, information overload, time pressure, and the need to aggregate and synthesize information from disparate sources. The ability to organize information in ways that facilitate the generation of effective diagnostic solutions is a distinguishing characteristic of expert physicians, suggesting that automated systems that organize clinical information in a similar manner may augment physicians' decision-making capabilities. In this paper, we describe the design and evaluation of a theoretically driven cognitive support system (CSS) that assists psychiatrists in their interpretation of clinical cases. The system highlights, and provides the means to navigate to, text that is organized in accordance with a set of diagnostically and therapeutically meaningful higher-level concepts. Methods and materials: To evaluate the interface, 16 psychiatry residents interpreted two clinical case scenarios, with and without the CSS. Think-aloud protocols captured during their interpretation of the cases were transcribed and analyzed qualitatively. In addition, the frequency and relative position of content related to key higher-level concepts in a verbal summary of the case were evaluated. In addition the transcripts from both groups were compared to an expert derived reference standard using latent semantic analysis (LSA). Results: Qualitative analysis showed that users of the system better attended to specific clinically important aspects of both cases when these were highlighted by the system, and revealed ways in which the system mediates hypotheses generation and evaluation. Analysis of the summary data showed differences in emphasis with and without the system. The LSA analysis suggested users of the system were more {"}expert-like{"} in their emphasis, and that cognitive support was more effective in the more complex case. Conclusions: Cognitive support impacts upon clinical comprehension. This appears to be largely helpful, but may also lead to neglect of information (such as the psychosocial history) that the system does not highlight. The results have implications for the design of CSSs for clinical narratives including the role of information organization and textual embellishments for more efficient clinical case presentation and comprehension.",
keywords = "Biomedical informatics, Clinical comprehension, Cognitive science, Cognitive support, Emergency psychiatry, Latent semantic analysis, Propositional analysis, Psychiatry, Verbal protocol analysis",
author = "Dalai, {Venkata V.} and Sana Khalid and Dinesh Gottipati and Thomas Kannampallil and Vineeth John and Brett Blatter and Vimla Patel and Trevor Cohen",
year = "2014",
month = "10",
day = "1",
doi = "10.1016/j.artmed.2014.08.002",
language = "English (US)",
volume = "62",
pages = "91--104",
journal = "Artificial Intelligence in Medicine",
issn = "0933-3657",
publisher = "Elsevier",
number = "2",

}

TY - JOUR

T1 - Evaluating the effects of cognitive support on psychiatric clinical comprehension

AU - Dalai, Venkata V.

AU - Khalid, Sana

AU - Gottipati, Dinesh

AU - Kannampallil, Thomas

AU - John, Vineeth

AU - Blatter, Brett

AU - Patel, Vimla

AU - Cohen, Trevor

PY - 2014/10/1

Y1 - 2014/10/1

N2 - Objective: Clinicians' attention is a precious resource, which in the current healthcare practice is consumed by the cognitive demands arising from complex patient conditions, information overload, time pressure, and the need to aggregate and synthesize information from disparate sources. The ability to organize information in ways that facilitate the generation of effective diagnostic solutions is a distinguishing characteristic of expert physicians, suggesting that automated systems that organize clinical information in a similar manner may augment physicians' decision-making capabilities. In this paper, we describe the design and evaluation of a theoretically driven cognitive support system (CSS) that assists psychiatrists in their interpretation of clinical cases. The system highlights, and provides the means to navigate to, text that is organized in accordance with a set of diagnostically and therapeutically meaningful higher-level concepts. Methods and materials: To evaluate the interface, 16 psychiatry residents interpreted two clinical case scenarios, with and without the CSS. Think-aloud protocols captured during their interpretation of the cases were transcribed and analyzed qualitatively. In addition, the frequency and relative position of content related to key higher-level concepts in a verbal summary of the case were evaluated. In addition the transcripts from both groups were compared to an expert derived reference standard using latent semantic analysis (LSA). Results: Qualitative analysis showed that users of the system better attended to specific clinically important aspects of both cases when these were highlighted by the system, and revealed ways in which the system mediates hypotheses generation and evaluation. Analysis of the summary data showed differences in emphasis with and without the system. The LSA analysis suggested users of the system were more "expert-like" in their emphasis, and that cognitive support was more effective in the more complex case. Conclusions: Cognitive support impacts upon clinical comprehension. This appears to be largely helpful, but may also lead to neglect of information (such as the psychosocial history) that the system does not highlight. The results have implications for the design of CSSs for clinical narratives including the role of information organization and textual embellishments for more efficient clinical case presentation and comprehension.

AB - Objective: Clinicians' attention is a precious resource, which in the current healthcare practice is consumed by the cognitive demands arising from complex patient conditions, information overload, time pressure, and the need to aggregate and synthesize information from disparate sources. The ability to organize information in ways that facilitate the generation of effective diagnostic solutions is a distinguishing characteristic of expert physicians, suggesting that automated systems that organize clinical information in a similar manner may augment physicians' decision-making capabilities. In this paper, we describe the design and evaluation of a theoretically driven cognitive support system (CSS) that assists psychiatrists in their interpretation of clinical cases. The system highlights, and provides the means to navigate to, text that is organized in accordance with a set of diagnostically and therapeutically meaningful higher-level concepts. Methods and materials: To evaluate the interface, 16 psychiatry residents interpreted two clinical case scenarios, with and without the CSS. Think-aloud protocols captured during their interpretation of the cases were transcribed and analyzed qualitatively. In addition, the frequency and relative position of content related to key higher-level concepts in a verbal summary of the case were evaluated. In addition the transcripts from both groups were compared to an expert derived reference standard using latent semantic analysis (LSA). Results: Qualitative analysis showed that users of the system better attended to specific clinically important aspects of both cases when these were highlighted by the system, and revealed ways in which the system mediates hypotheses generation and evaluation. Analysis of the summary data showed differences in emphasis with and without the system. The LSA analysis suggested users of the system were more "expert-like" in their emphasis, and that cognitive support was more effective in the more complex case. Conclusions: Cognitive support impacts upon clinical comprehension. This appears to be largely helpful, but may also lead to neglect of information (such as the psychosocial history) that the system does not highlight. The results have implications for the design of CSSs for clinical narratives including the role of information organization and textual embellishments for more efficient clinical case presentation and comprehension.

KW - Biomedical informatics

KW - Clinical comprehension

KW - Cognitive science

KW - Cognitive support

KW - Emergency psychiatry

KW - Latent semantic analysis

KW - Propositional analysis

KW - Psychiatry

KW - Verbal protocol analysis

UR - http://www.scopus.com/inward/record.url?scp=84910013114&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84910013114&partnerID=8YFLogxK

U2 - 10.1016/j.artmed.2014.08.002

DO - 10.1016/j.artmed.2014.08.002

M3 - Article

VL - 62

SP - 91

EP - 104

JO - Artificial Intelligence in Medicine

JF - Artificial Intelligence in Medicine

SN - 0933-3657

IS - 2

ER -