Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes

Leah Friedman, Erin Walker, Ruixue Liu, Erin T. Solovey

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As non-invasive neuroimaging techniques become less expensive and more portable, we have the capability to monitor brain activity during various computer activities. This provides an opportunity to integrate brain data with computer log data to develop models of cognitive processes. These models can be used to continually assess an individual’s changing cognitive state and develop adaptive human-computer interfaces. As a step in this direction, we have conducted a study using functional near-infrared spectroscopy (fNIRS) during the Sustained Attention to Response Task (SART) paradigm, which has been used in prior work to elicit mind wandering and to explore response inhibition. The goal with this is to determine whether fNIRS data can be used as a predictor of errors on the task. This would have implications for detecting similar cognitive processes in more realistic tasks, such as using a personal learning environment. Additionally, this study aims to test individual differences by correlating objective behavioral data and subjective self reports with activity in the medial prefrontal cortex (mPFC), associated with the brain’s default mode network (DMN). We observed significant differences in the mPFC between periods prior to task error and periods prior to a correct response. These differences were particularly apparent amongst those individuals who performed poorly on the SART task and those who reported drowsiness. In line with previous work, these findings indicate an opportunity to detect and correct attentional shifts in individuals who need it most.

Original languageEnglish (US)
Title of host publicationProceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450360722
DOIs
StatePublished - Oct 16 2018
Externally publishedYes
Event2018 Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018 - Boulder, United States
Duration: Oct 16 2018 → …

Publication series

NameProceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018

Conference

Conference2018 Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018
CountryUnited States
CityBoulder
Period10/16/18 → …

Fingerprint

Neuroimaging
Datalog
Brain
Near infrared spectroscopy
Near-infrared Spectroscopy
Cortex
Human-computer Interface
Interfaces (computer)
Individual Differences
Learning Environment
Predictors
Monitor
Paradigm
Integrate
Line
Model

Keywords

  • FNIRS
  • Functional near-infrared spectroscopy
  • SART

ASJC Scopus subject areas

  • Modeling and Simulation

Cite this

Friedman, L., Walker, E., Liu, R., & Solovey, E. T. (2018). Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes. In Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018 (Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018). Association for Computing Machinery, Inc. https://doi.org/10.1145/3279810.3279854

Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes. / Friedman, Leah; Walker, Erin; Liu, Ruixue; Solovey, Erin T.

Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018. Association for Computing Machinery, Inc, 2018. (Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Friedman, L, Walker, E, Liu, R & Solovey, ET 2018, Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes. in Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018. Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018, Association for Computing Machinery, Inc, 2018 Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018, Boulder, United States, 10/16/18. https://doi.org/10.1145/3279810.3279854
Friedman L, Walker E, Liu R, Solovey ET. Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes. In Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018. Association for Computing Machinery, Inc. 2018. (Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018). https://doi.org/10.1145/3279810.3279854
Friedman, Leah ; Walker, Erin ; Liu, Ruixue ; Solovey, Erin T. / Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes. Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018. Association for Computing Machinery, Inc, 2018. (Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018).
@inproceedings{9ab26be031dd40b98001e26cccf58e96,
title = "Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes",
abstract = "As non-invasive neuroimaging techniques become less expensive and more portable, we have the capability to monitor brain activity during various computer activities. This provides an opportunity to integrate brain data with computer log data to develop models of cognitive processes. These models can be used to continually assess an individual’s changing cognitive state and develop adaptive human-computer interfaces. As a step in this direction, we have conducted a study using functional near-infrared spectroscopy (fNIRS) during the Sustained Attention to Response Task (SART) paradigm, which has been used in prior work to elicit mind wandering and to explore response inhibition. The goal with this is to determine whether fNIRS data can be used as a predictor of errors on the task. This would have implications for detecting similar cognitive processes in more realistic tasks, such as using a personal learning environment. Additionally, this study aims to test individual differences by correlating objective behavioral data and subjective self reports with activity in the medial prefrontal cortex (mPFC), associated with the brain’s default mode network (DMN). We observed significant differences in the mPFC between periods prior to task error and periods prior to a correct response. These differences were particularly apparent amongst those individuals who performed poorly on the SART task and those who reported drowsiness. In line with previous work, these findings indicate an opportunity to detect and correct attentional shifts in individuals who need it most.",
keywords = "FNIRS, Functional near-infrared spectroscopy, SART",
author = "Leah Friedman and Erin Walker and Ruixue Liu and Solovey, {Erin T.}",
year = "2018",
month = "10",
day = "16",
doi = "10.1145/3279810.3279854",
language = "English (US)",
series = "Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018",
publisher = "Association for Computing Machinery, Inc",
booktitle = "Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018",

}

TY - GEN

T1 - Integrating non-invasive neuroimaging and computer log data to improve understanding of cognitive processes

AU - Friedman, Leah

AU - Walker, Erin

AU - Liu, Ruixue

AU - Solovey, Erin T.

PY - 2018/10/16

Y1 - 2018/10/16

N2 - As non-invasive neuroimaging techniques become less expensive and more portable, we have the capability to monitor brain activity during various computer activities. This provides an opportunity to integrate brain data with computer log data to develop models of cognitive processes. These models can be used to continually assess an individual’s changing cognitive state and develop adaptive human-computer interfaces. As a step in this direction, we have conducted a study using functional near-infrared spectroscopy (fNIRS) during the Sustained Attention to Response Task (SART) paradigm, which has been used in prior work to elicit mind wandering and to explore response inhibition. The goal with this is to determine whether fNIRS data can be used as a predictor of errors on the task. This would have implications for detecting similar cognitive processes in more realistic tasks, such as using a personal learning environment. Additionally, this study aims to test individual differences by correlating objective behavioral data and subjective self reports with activity in the medial prefrontal cortex (mPFC), associated with the brain’s default mode network (DMN). We observed significant differences in the mPFC between periods prior to task error and periods prior to a correct response. These differences were particularly apparent amongst those individuals who performed poorly on the SART task and those who reported drowsiness. In line with previous work, these findings indicate an opportunity to detect and correct attentional shifts in individuals who need it most.

AB - As non-invasive neuroimaging techniques become less expensive and more portable, we have the capability to monitor brain activity during various computer activities. This provides an opportunity to integrate brain data with computer log data to develop models of cognitive processes. These models can be used to continually assess an individual’s changing cognitive state and develop adaptive human-computer interfaces. As a step in this direction, we have conducted a study using functional near-infrared spectroscopy (fNIRS) during the Sustained Attention to Response Task (SART) paradigm, which has been used in prior work to elicit mind wandering and to explore response inhibition. The goal with this is to determine whether fNIRS data can be used as a predictor of errors on the task. This would have implications for detecting similar cognitive processes in more realistic tasks, such as using a personal learning environment. Additionally, this study aims to test individual differences by correlating objective behavioral data and subjective self reports with activity in the medial prefrontal cortex (mPFC), associated with the brain’s default mode network (DMN). We observed significant differences in the mPFC between periods prior to task error and periods prior to a correct response. These differences were particularly apparent amongst those individuals who performed poorly on the SART task and those who reported drowsiness. In line with previous work, these findings indicate an opportunity to detect and correct attentional shifts in individuals who need it most.

KW - FNIRS

KW - Functional near-infrared spectroscopy

KW - SART

UR - http://www.scopus.com/inward/record.url?scp=85058267883&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058267883&partnerID=8YFLogxK

U2 - 10.1145/3279810.3279854

DO - 10.1145/3279810.3279854

M3 - Conference contribution

AN - SCOPUS:85058267883

T3 - Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018

BT - Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data, MCPMD 2018

PB - Association for Computing Machinery, Inc

ER -