Computational heterogeneity in the human mesencephalic dopamine system

Kimberlee McClure, Terry Lohrenz, Krystle A. Bartley, P. Read Montague

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Recent evidence in animals has indicated that the mesencephalic dopamine system is heterogeneous anatomically, molecularly, and functionally, and it has been suggested that the dopamine system comprises distinct functional systems. Identifying and characterizing these systems in humans will have widespread ramifications for understanding drug addiction and mental health disorders. Model-based studies in humans have suggested an analogous computational heterogeneity, in which dopaminergic targets in striatum encode both experience-based learning signals and counterfactual learning signals that are based on hypothetical information. We used brainstem-tailored fMRI to identify mesencephalic sources of experiential and counterfactual learning signals. Participants completed a decision-making task based on investing in markets. This sequential investment task generated experience-based learning signals, in the form of temporal difference (TD) reward prediction errors, and counterfactual learning signals, in the form of "fictive errors." Fictive errors are reinforcement learning signals based on hypothetical information about "what could have been." An additional learning signal was constructed to be relatable to a motivational salience signal. Blood oxygenation level dependent responses in regions of substantia nigra (SN) and ventral tegmental area (VTA), where dopamine neurons are located, coded for TD and fictive errors, and additionally were related to the motivational salience signal these results are highly consistent with animal electrophysiology and provide direct evidence that human SN and VTA heterogeneously handle important reward-harvesting computations.

Original languageEnglish (US)
Pages (from-to)747-756
Number of pages10
JournalCognitive, Affective and Behavioral Neuroscience
Volume13
Issue number4
DOIs
StatePublished - Dec 1 2013
Externally publishedYes

Fingerprint

Dopamine
Learning
Ventral Tegmental Area
Substantia Nigra
Reward
Problem-Based Learning
Electrophysiology
Dopaminergic Neurons
Mental Disorders
Brain Stem
Substance-Related Disorders
Decision Making
Mental Health
Magnetic Resonance Imaging

Keywords

  • Brainstem fMRI
  • Decision making
  • Dopamine
  • Reinforcement learning
  • Reward
  • SN
  • VTA

ASJC Scopus subject areas

  • Behavioral Neuroscience
  • Cognitive Neuroscience

Cite this

Computational heterogeneity in the human mesencephalic dopamine system. / McClure, Kimberlee; Lohrenz, Terry; Bartley, Krystle A.; Montague, P. Read.

In: Cognitive, Affective and Behavioral Neuroscience, Vol. 13, No. 4, 01.12.2013, p. 747-756.

Research output: Contribution to journalArticle

McClure, Kimberlee ; Lohrenz, Terry ; Bartley, Krystle A. ; Montague, P. Read. / Computational heterogeneity in the human mesencephalic dopamine system. In: Cognitive, Affective and Behavioral Neuroscience. 2013 ; Vol. 13, No. 4. pp. 747-756.
@article{7ee09b8afa39421b8f2868acb6341810,
title = "Computational heterogeneity in the human mesencephalic dopamine system",
abstract = "Recent evidence in animals has indicated that the mesencephalic dopamine system is heterogeneous anatomically, molecularly, and functionally, and it has been suggested that the dopamine system comprises distinct functional systems. Identifying and characterizing these systems in humans will have widespread ramifications for understanding drug addiction and mental health disorders. Model-based studies in humans have suggested an analogous computational heterogeneity, in which dopaminergic targets in striatum encode both experience-based learning signals and counterfactual learning signals that are based on hypothetical information. We used brainstem-tailored fMRI to identify mesencephalic sources of experiential and counterfactual learning signals. Participants completed a decision-making task based on investing in markets. This sequential investment task generated experience-based learning signals, in the form of temporal difference (TD) reward prediction errors, and counterfactual learning signals, in the form of {"}fictive errors.{"} Fictive errors are reinforcement learning signals based on hypothetical information about {"}what could have been.{"} An additional learning signal was constructed to be relatable to a motivational salience signal. Blood oxygenation level dependent responses in regions of substantia nigra (SN) and ventral tegmental area (VTA), where dopamine neurons are located, coded for TD and fictive errors, and additionally were related to the motivational salience signal these results are highly consistent with animal electrophysiology and provide direct evidence that human SN and VTA heterogeneously handle important reward-harvesting computations.",
keywords = "Brainstem fMRI, Decision making, Dopamine, Reinforcement learning, Reward, SN, VTA",
author = "Kimberlee McClure and Terry Lohrenz and Bartley, {Krystle A.} and Montague, {P. Read}",
year = "2013",
month = "12",
day = "1",
doi = "10.3758/s13415-013-0191-5",
language = "English (US)",
volume = "13",
pages = "747--756",
journal = "Cognitive, Affective and Behavioral Neuroscience",
issn = "1530-7026",
publisher = "Springer New York",
number = "4",

}

TY - JOUR

T1 - Computational heterogeneity in the human mesencephalic dopamine system

AU - McClure, Kimberlee

AU - Lohrenz, Terry

AU - Bartley, Krystle A.

AU - Montague, P. Read

PY - 2013/12/1

Y1 - 2013/12/1

N2 - Recent evidence in animals has indicated that the mesencephalic dopamine system is heterogeneous anatomically, molecularly, and functionally, and it has been suggested that the dopamine system comprises distinct functional systems. Identifying and characterizing these systems in humans will have widespread ramifications for understanding drug addiction and mental health disorders. Model-based studies in humans have suggested an analogous computational heterogeneity, in which dopaminergic targets in striatum encode both experience-based learning signals and counterfactual learning signals that are based on hypothetical information. We used brainstem-tailored fMRI to identify mesencephalic sources of experiential and counterfactual learning signals. Participants completed a decision-making task based on investing in markets. This sequential investment task generated experience-based learning signals, in the form of temporal difference (TD) reward prediction errors, and counterfactual learning signals, in the form of "fictive errors." Fictive errors are reinforcement learning signals based on hypothetical information about "what could have been." An additional learning signal was constructed to be relatable to a motivational salience signal. Blood oxygenation level dependent responses in regions of substantia nigra (SN) and ventral tegmental area (VTA), where dopamine neurons are located, coded for TD and fictive errors, and additionally were related to the motivational salience signal these results are highly consistent with animal electrophysiology and provide direct evidence that human SN and VTA heterogeneously handle important reward-harvesting computations.

AB - Recent evidence in animals has indicated that the mesencephalic dopamine system is heterogeneous anatomically, molecularly, and functionally, and it has been suggested that the dopamine system comprises distinct functional systems. Identifying and characterizing these systems in humans will have widespread ramifications for understanding drug addiction and mental health disorders. Model-based studies in humans have suggested an analogous computational heterogeneity, in which dopaminergic targets in striatum encode both experience-based learning signals and counterfactual learning signals that are based on hypothetical information. We used brainstem-tailored fMRI to identify mesencephalic sources of experiential and counterfactual learning signals. Participants completed a decision-making task based on investing in markets. This sequential investment task generated experience-based learning signals, in the form of temporal difference (TD) reward prediction errors, and counterfactual learning signals, in the form of "fictive errors." Fictive errors are reinforcement learning signals based on hypothetical information about "what could have been." An additional learning signal was constructed to be relatable to a motivational salience signal. Blood oxygenation level dependent responses in regions of substantia nigra (SN) and ventral tegmental area (VTA), where dopamine neurons are located, coded for TD and fictive errors, and additionally were related to the motivational salience signal these results are highly consistent with animal electrophysiology and provide direct evidence that human SN and VTA heterogeneously handle important reward-harvesting computations.

KW - Brainstem fMRI

KW - Decision making

KW - Dopamine

KW - Reinforcement learning

KW - Reward

KW - SN

KW - VTA

UR - http://www.scopus.com/inward/record.url?scp=84891421186&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84891421186&partnerID=8YFLogxK

U2 - 10.3758/s13415-013-0191-5

DO - 10.3758/s13415-013-0191-5

M3 - Article

C2 - 23943512

AN - SCOPUS:84891421186

VL - 13

SP - 747

EP - 756

JO - Cognitive, Affective and Behavioral Neuroscience

JF - Cognitive, Affective and Behavioral Neuroscience

SN - 1530-7026

IS - 4

ER -