Hierarchical control over effortful behavior by rodent medial frontal cortex

A computational model

Clay B. Holroyd, Samuel McClure

Research output: Contribution to journalArticle

78 Citations (Scopus)

Abstract

The anterior cingulate cortex (ACC) has been the focus of intense research interest in recent years. Although separate theories relate ACC function variously to conflict monitoring, reward processing, action selection, decision making, and more, damage to the ACC mostly spares performance on tasks that exercise these functions, indicating that they are not in fact unique to the ACC. Further, most theories do not address the most salient consequence of ACC damage: impoverished action generation in the presence of normal motor ability. In this study we develop a computational model of the rodent medial prefrontal cortex that accounts for the behavioral sequelae of ACC damage, unifies many of the cognitive functions attributed to it, and provides a solution to an outstanding question in cognitive control research-how the control system determines and motivates what tasks to perform. The theory derives from recent developments in the formal study of hierarchical control and learning that highlight computational efficiencies afforded when collections of actions are represented based on their conjoint goals. According to this position, the ACC utilizes reward information to select tasks that are then accomplished through top-down control over action selection by the striatum. Computational simulations capture animal lesion data that implicate the medial prefrontal cortex in regulating physical and cognitive effort. Overall, this theory provides a unifying theoretical framework for understanding the ACC in terms of the pivotal role it plays in the hierarchical organization of effortful behavior.

Original languageEnglish (US)
Pages (from-to)54-83
Number of pages30
JournalPsychological Review
Volume122
Issue number1
DOIs
StatePublished - 2015
Externally publishedYes

Fingerprint

Gyrus Cinguli
Frontal Lobe
Rodentia
Prefrontal Cortex
Reward
Physical Exertion
Aptitude
Cortex
Rodent
Computational Model
Task Performance and Analysis
Research
Cognition
Decision Making
Learning
Efficiency

Keywords

  • Anterior cingulate cortex
  • Cognitive control
  • Effort
  • Hierarchical reinforcement learning
  • Prelimbic cortex

ASJC Scopus subject areas

  • Psychology(all)
  • History and Philosophy of Science

Cite this

Hierarchical control over effortful behavior by rodent medial frontal cortex : A computational model. / Holroyd, Clay B.; McClure, Samuel.

In: Psychological Review, Vol. 122, No. 1, 2015, p. 54-83.

Research output: Contribution to journalArticle

@article{25b532da05a24b88911ed73576a2649f,
title = "Hierarchical control over effortful behavior by rodent medial frontal cortex: A computational model",
abstract = "The anterior cingulate cortex (ACC) has been the focus of intense research interest in recent years. Although separate theories relate ACC function variously to conflict monitoring, reward processing, action selection, decision making, and more, damage to the ACC mostly spares performance on tasks that exercise these functions, indicating that they are not in fact unique to the ACC. Further, most theories do not address the most salient consequence of ACC damage: impoverished action generation in the presence of normal motor ability. In this study we develop a computational model of the rodent medial prefrontal cortex that accounts for the behavioral sequelae of ACC damage, unifies many of the cognitive functions attributed to it, and provides a solution to an outstanding question in cognitive control research-how the control system determines and motivates what tasks to perform. The theory derives from recent developments in the formal study of hierarchical control and learning that highlight computational efficiencies afforded when collections of actions are represented based on their conjoint goals. According to this position, the ACC utilizes reward information to select tasks that are then accomplished through top-down control over action selection by the striatum. Computational simulations capture animal lesion data that implicate the medial prefrontal cortex in regulating physical and cognitive effort. Overall, this theory provides a unifying theoretical framework for understanding the ACC in terms of the pivotal role it plays in the hierarchical organization of effortful behavior.",
keywords = "Anterior cingulate cortex, Cognitive control, Effort, Hierarchical reinforcement learning, Prelimbic cortex",
author = "Holroyd, {Clay B.} and Samuel McClure",
year = "2015",
doi = "10.1037/a0038339",
language = "English (US)",
volume = "122",
pages = "54--83",
journal = "Psychological Review",
issn = "0033-295X",
publisher = "American Psychological Association Inc.",
number = "1",

}

TY - JOUR

T1 - Hierarchical control over effortful behavior by rodent medial frontal cortex

T2 - A computational model

AU - Holroyd, Clay B.

AU - McClure, Samuel

PY - 2015

Y1 - 2015

N2 - The anterior cingulate cortex (ACC) has been the focus of intense research interest in recent years. Although separate theories relate ACC function variously to conflict monitoring, reward processing, action selection, decision making, and more, damage to the ACC mostly spares performance on tasks that exercise these functions, indicating that they are not in fact unique to the ACC. Further, most theories do not address the most salient consequence of ACC damage: impoverished action generation in the presence of normal motor ability. In this study we develop a computational model of the rodent medial prefrontal cortex that accounts for the behavioral sequelae of ACC damage, unifies many of the cognitive functions attributed to it, and provides a solution to an outstanding question in cognitive control research-how the control system determines and motivates what tasks to perform. The theory derives from recent developments in the formal study of hierarchical control and learning that highlight computational efficiencies afforded when collections of actions are represented based on their conjoint goals. According to this position, the ACC utilizes reward information to select tasks that are then accomplished through top-down control over action selection by the striatum. Computational simulations capture animal lesion data that implicate the medial prefrontal cortex in regulating physical and cognitive effort. Overall, this theory provides a unifying theoretical framework for understanding the ACC in terms of the pivotal role it plays in the hierarchical organization of effortful behavior.

AB - The anterior cingulate cortex (ACC) has been the focus of intense research interest in recent years. Although separate theories relate ACC function variously to conflict monitoring, reward processing, action selection, decision making, and more, damage to the ACC mostly spares performance on tasks that exercise these functions, indicating that they are not in fact unique to the ACC. Further, most theories do not address the most salient consequence of ACC damage: impoverished action generation in the presence of normal motor ability. In this study we develop a computational model of the rodent medial prefrontal cortex that accounts for the behavioral sequelae of ACC damage, unifies many of the cognitive functions attributed to it, and provides a solution to an outstanding question in cognitive control research-how the control system determines and motivates what tasks to perform. The theory derives from recent developments in the formal study of hierarchical control and learning that highlight computational efficiencies afforded when collections of actions are represented based on their conjoint goals. According to this position, the ACC utilizes reward information to select tasks that are then accomplished through top-down control over action selection by the striatum. Computational simulations capture animal lesion data that implicate the medial prefrontal cortex in regulating physical and cognitive effort. Overall, this theory provides a unifying theoretical framework for understanding the ACC in terms of the pivotal role it plays in the hierarchical organization of effortful behavior.

KW - Anterior cingulate cortex

KW - Cognitive control

KW - Effort

KW - Hierarchical reinforcement learning

KW - Prelimbic cortex

UR - http://www.scopus.com/inward/record.url?scp=84923002419&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84923002419&partnerID=8YFLogxK

U2 - 10.1037/a0038339

DO - 10.1037/a0038339

M3 - Article

VL - 122

SP - 54

EP - 83

JO - Psychological Review

JF - Psychological Review

SN - 0033-295X

IS - 1

ER -