Neural networks supporting audiovisual integration for speech: A large-scale lesion study

Gregory Hickok, Corianne Reddy, William Matchin, Alexandra Basilakos, Julia Cai, Sara Pillay, Michelle Ferrill, Soren Mickelsen, Steven W. Anderson, Tracy Love, Jeffrey Binder, Julius Fridriksson

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Auditory and visual speech information are often strongly integrated resulting in perceptual enhancements for audiovisual (AV) speech over audio alone and sometimes yielding compelling illusory fusion percepts when AV cues are mismatched, the McGurk-MacDonald effect. Previous research has identified three candidate regions thought to be critical for AV speech integration: the posterior superior temporal sulcus (STS), early auditory cortex, and the posterior inferior frontal gyrus. We assess the causal involvement of these regions (and others) in the first large-scale (N = 100) lesion-based study of AV speech integration. Two primary findings emerged. First, behavioral performance and lesion maps for AV enhancement and illusory fusion measures indicate that classic metrics of AV speech integration are not necessarily measuring the same process. Second, lesions involving superior temporal auditory, lateral occipital visual, and multisensory zones in the STS are the most disruptive to AV speech integration. Further, when AV speech integration fails, the nature of the failure—auditory vs visual capture—can be predicted from the location of the lesions. These findings show that AV speech processing is supported by unimodal auditory and visual cortices as well as multimodal regions such as the STS at their boundary. Motor related frontal regions do not appear to play a role in AV speech integration.

Original languageEnglish (US)
Pages (from-to)360-371
Number of pages12
JournalCortex
Volume103
DOIs
StatePublished - Jun 1 2018

Fingerprint

Temporal Lobe
Auditory Cortex
Visual Cortex
Prefrontal Cortex
Cues
Research

Keywords

  • Speech audiovisual stroke language aphasia

ASJC Scopus subject areas

  • Neuropsychology and Physiological Psychology
  • Experimental and Cognitive Psychology
  • Cognitive Neuroscience

Cite this

Hickok, G., Reddy, C., Matchin, W., Basilakos, A., Cai, J., Pillay, S., ... Fridriksson, J. (2018). Neural networks supporting audiovisual integration for speech: A large-scale lesion study. Cortex, 103, 360-371. https://doi.org/10.1016/j.cortex.2018.03.030

Neural networks supporting audiovisual integration for speech : A large-scale lesion study. / Hickok, Gregory; Reddy, Corianne; Matchin, William; Basilakos, Alexandra; Cai, Julia; Pillay, Sara; Ferrill, Michelle; Mickelsen, Soren; Anderson, Steven W.; Love, Tracy; Binder, Jeffrey; Fridriksson, Julius.

In: Cortex, Vol. 103, 01.06.2018, p. 360-371.

Research output: Contribution to journalArticle

Hickok, G, Reddy, C, Matchin, W, Basilakos, A, Cai, J, Pillay, S, Ferrill, M, Mickelsen, S, Anderson, SW, Love, T, Binder, J & Fridriksson, J 2018, 'Neural networks supporting audiovisual integration for speech: A large-scale lesion study', Cortex, vol. 103, pp. 360-371. https://doi.org/10.1016/j.cortex.2018.03.030
Hickok, Gregory ; Reddy, Corianne ; Matchin, William ; Basilakos, Alexandra ; Cai, Julia ; Pillay, Sara ; Ferrill, Michelle ; Mickelsen, Soren ; Anderson, Steven W. ; Love, Tracy ; Binder, Jeffrey ; Fridriksson, Julius. / Neural networks supporting audiovisual integration for speech : A large-scale lesion study. In: Cortex. 2018 ; Vol. 103. pp. 360-371.
@article{6d70f724034540f89432dc0499e19def,
title = "Neural networks supporting audiovisual integration for speech: A large-scale lesion study",
abstract = "Auditory and visual speech information are often strongly integrated resulting in perceptual enhancements for audiovisual (AV) speech over audio alone and sometimes yielding compelling illusory fusion percepts when AV cues are mismatched, the McGurk-MacDonald effect. Previous research has identified three candidate regions thought to be critical for AV speech integration: the posterior superior temporal sulcus (STS), early auditory cortex, and the posterior inferior frontal gyrus. We assess the causal involvement of these regions (and others) in the first large-scale (N = 100) lesion-based study of AV speech integration. Two primary findings emerged. First, behavioral performance and lesion maps for AV enhancement and illusory fusion measures indicate that classic metrics of AV speech integration are not necessarily measuring the same process. Second, lesions involving superior temporal auditory, lateral occipital visual, and multisensory zones in the STS are the most disruptive to AV speech integration. Further, when AV speech integration fails, the nature of the failure—auditory vs visual capture—can be predicted from the location of the lesions. These findings show that AV speech processing is supported by unimodal auditory and visual cortices as well as multimodal regions such as the STS at their boundary. Motor related frontal regions do not appear to play a role in AV speech integration.",
keywords = "Speech audiovisual stroke language aphasia",
author = "Gregory Hickok and Corianne Reddy and William Matchin and Alexandra Basilakos and Julia Cai and Sara Pillay and Michelle Ferrill and Soren Mickelsen and Anderson, {Steven W.} and Tracy Love and Jeffrey Binder and Julius Fridriksson",
year = "2018",
month = "6",
day = "1",
doi = "10.1016/j.cortex.2018.03.030",
language = "English (US)",
volume = "103",
pages = "360--371",
journal = "Cortex",
issn = "0010-9452",
publisher = "Masson SpA",

}

TY - JOUR

T1 - Neural networks supporting audiovisual integration for speech

T2 - A large-scale lesion study

AU - Hickok, Gregory

AU - Reddy, Corianne

AU - Matchin, William

AU - Basilakos, Alexandra

AU - Cai, Julia

AU - Pillay, Sara

AU - Ferrill, Michelle

AU - Mickelsen, Soren

AU - Anderson, Steven W.

AU - Love, Tracy

AU - Binder, Jeffrey

AU - Fridriksson, Julius

PY - 2018/6/1

Y1 - 2018/6/1

N2 - Auditory and visual speech information are often strongly integrated resulting in perceptual enhancements for audiovisual (AV) speech over audio alone and sometimes yielding compelling illusory fusion percepts when AV cues are mismatched, the McGurk-MacDonald effect. Previous research has identified three candidate regions thought to be critical for AV speech integration: the posterior superior temporal sulcus (STS), early auditory cortex, and the posterior inferior frontal gyrus. We assess the causal involvement of these regions (and others) in the first large-scale (N = 100) lesion-based study of AV speech integration. Two primary findings emerged. First, behavioral performance and lesion maps for AV enhancement and illusory fusion measures indicate that classic metrics of AV speech integration are not necessarily measuring the same process. Second, lesions involving superior temporal auditory, lateral occipital visual, and multisensory zones in the STS are the most disruptive to AV speech integration. Further, when AV speech integration fails, the nature of the failure—auditory vs visual capture—can be predicted from the location of the lesions. These findings show that AV speech processing is supported by unimodal auditory and visual cortices as well as multimodal regions such as the STS at their boundary. Motor related frontal regions do not appear to play a role in AV speech integration.

AB - Auditory and visual speech information are often strongly integrated resulting in perceptual enhancements for audiovisual (AV) speech over audio alone and sometimes yielding compelling illusory fusion percepts when AV cues are mismatched, the McGurk-MacDonald effect. Previous research has identified three candidate regions thought to be critical for AV speech integration: the posterior superior temporal sulcus (STS), early auditory cortex, and the posterior inferior frontal gyrus. We assess the causal involvement of these regions (and others) in the first large-scale (N = 100) lesion-based study of AV speech integration. Two primary findings emerged. First, behavioral performance and lesion maps for AV enhancement and illusory fusion measures indicate that classic metrics of AV speech integration are not necessarily measuring the same process. Second, lesions involving superior temporal auditory, lateral occipital visual, and multisensory zones in the STS are the most disruptive to AV speech integration. Further, when AV speech integration fails, the nature of the failure—auditory vs visual capture—can be predicted from the location of the lesions. These findings show that AV speech processing is supported by unimodal auditory and visual cortices as well as multimodal regions such as the STS at their boundary. Motor related frontal regions do not appear to play a role in AV speech integration.

KW - Speech audiovisual stroke language aphasia

UR - http://www.scopus.com/inward/record.url?scp=85046169918&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85046169918&partnerID=8YFLogxK

U2 - 10.1016/j.cortex.2018.03.030

DO - 10.1016/j.cortex.2018.03.030

M3 - Article

C2 - 29705718

AN - SCOPUS:85046169918

VL - 103

SP - 360

EP - 371

JO - Cortex

JF - Cortex

SN - 0010-9452

ER -