Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning

Ian Ballard, Eric M. Miller, Steven T. Piantadosi, Noah D. Goodman, Samuel McClure

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Humans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning or by using explicit reasoning. We tested if the striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. Using functional magnetic resonance imaging during a categorization task, we show that striatal responses to feedback scale with a "surprise" signal derived from a Bayesian rule-learning model and are inconsistent with RL prediction error. We also find that striatum and caudal inferior frontal sulcus (cIFS) are involved in updating the likelihood of discriminative rules. We conclude that the striatum, in cooperation with the cIFS, is involved in updating the values assigned to categorization rules when people learn using explicit reasoning.

Original languageEnglish (US)
Pages (from-to)3965-3975
Number of pages11
JournalCerebral cortex (New York, N.Y. : 1991)
Volume28
Issue number11
DOIs
StatePublished - Nov 1 2018

Fingerprint

Prefrontal Cortex
Reward
Learning
Corpus Striatum
Magnetic Resonance Imaging
Reinforcement (Psychology)

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Cellular and Molecular Neuroscience

Cite this

Beyond Reward Prediction Errors : Human Striatum Updates Rule Values During Learning. / Ballard, Ian; Miller, Eric M.; Piantadosi, Steven T.; Goodman, Noah D.; McClure, Samuel.

In: Cerebral cortex (New York, N.Y. : 1991), Vol. 28, No. 11, 01.11.2018, p. 3965-3975.

Research output: Contribution to journalArticle

Ballard, Ian ; Miller, Eric M. ; Piantadosi, Steven T. ; Goodman, Noah D. ; McClure, Samuel. / Beyond Reward Prediction Errors : Human Striatum Updates Rule Values During Learning. In: Cerebral cortex (New York, N.Y. : 1991). 2018 ; Vol. 28, No. 11. pp. 3965-3975.
@article{6073424dbf724cfaae29b332a57b2b00,
title = "Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning",
abstract = "Humans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning or by using explicit reasoning. We tested if the striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. Using functional magnetic resonance imaging during a categorization task, we show that striatal responses to feedback scale with a {"}surprise{"} signal derived from a Bayesian rule-learning model and are inconsistent with RL prediction error. We also find that striatum and caudal inferior frontal sulcus (cIFS) are involved in updating the likelihood of discriminative rules. We conclude that the striatum, in cooperation with the cIFS, is involved in updating the values assigned to categorization rules when people learn using explicit reasoning.",
author = "Ian Ballard and Miller, {Eric M.} and Piantadosi, {Steven T.} and Goodman, {Noah D.} and Samuel McClure",
year = "2018",
month = "11",
day = "1",
doi = "10.1093/cercor/bhx259",
language = "English (US)",
volume = "28",
pages = "3965--3975",
journal = "Cerebral Cortex",
issn = "1047-3211",
publisher = "Oxford University Press",
number = "11",

}

TY - JOUR

T1 - Beyond Reward Prediction Errors

T2 - Human Striatum Updates Rule Values During Learning

AU - Ballard, Ian

AU - Miller, Eric M.

AU - Piantadosi, Steven T.

AU - Goodman, Noah D.

AU - McClure, Samuel

PY - 2018/11/1

Y1 - 2018/11/1

N2 - Humans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning or by using explicit reasoning. We tested if the striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. Using functional magnetic resonance imaging during a categorization task, we show that striatal responses to feedback scale with a "surprise" signal derived from a Bayesian rule-learning model and are inconsistent with RL prediction error. We also find that striatum and caudal inferior frontal sulcus (cIFS) are involved in updating the likelihood of discriminative rules. We conclude that the striatum, in cooperation with the cIFS, is involved in updating the values assigned to categorization rules when people learn using explicit reasoning.

AB - Humans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning or by using explicit reasoning. We tested if the striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. Using functional magnetic resonance imaging during a categorization task, we show that striatal responses to feedback scale with a "surprise" signal derived from a Bayesian rule-learning model and are inconsistent with RL prediction error. We also find that striatum and caudal inferior frontal sulcus (cIFS) are involved in updating the likelihood of discriminative rules. We conclude that the striatum, in cooperation with the cIFS, is involved in updating the values assigned to categorization rules when people learn using explicit reasoning.

UR - http://www.scopus.com/inward/record.url?scp=85052631397&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052631397&partnerID=8YFLogxK

U2 - 10.1093/cercor/bhx259

DO - 10.1093/cercor/bhx259

M3 - Article

C2 - 29040494

AN - SCOPUS:85052631397

VL - 28

SP - 3965

EP - 3975

JO - Cerebral Cortex

JF - Cerebral Cortex

SN - 1047-3211

IS - 11

ER -