Visual capture of a stereo sound: Interactions between cue reliability, sound localization variability, and cross-modal bias

Christopher Montagne, Yi Zhou

    Research output: Contribution to journalArticle

    3 Citations (Scopus)

    Abstract

    Multisensory interactions involve coordination and sometimes competition between multiple senses. Vision usually dominates audition in spatial judgments when light and sound stimuli are presented from two different physical locations. This study investigated the influence of vision on the perceived location of a phantom sound source placed in a stereo sound field using a pair of loudspeakers emitting identical signals that were delayed or attenuated relative to each other. Results show that although a similar horizontal range (+/-45°) was reported for timing-modulated and level-modulated signals, listeners' localization performance showed greater variability for the timing signals. When visual stimuli were presented simultaneously with the auditory stimuli, listeners showed stronger visual bias for timing-modulated signals than level-modulated and single-speaker control signals. Trial-to-trial errors remained relatively stable over time, suggesting that sound localization uncertainty has an immediate and long-lasting effect on the across-modal bias. Binaural signal analyses further reveal that interaural differences of time and intensity - the two primary cues for sound localization in the azimuthal plane - are inherently more ambiguous for signals placed using timing. These results suggest that binaural ambiguity is intrinsically linked with localization variability and the strength of cross-modal bias in sound localization.

    Original languageEnglish (US)
    Pages (from-to)471-485
    Number of pages15
    JournalJournal of the Acoustical Society of America
    Volume140
    Issue number1
    DOIs
    StatePublished - Jul 1 2016

    Fingerprint

    sound localization
    cues
    acoustics
    time measurement
    interactions
    auditory stimuli
    visual stimuli
    loudspeakers
    sound fields
    Interaction
    Sound
    Sound Localization
    ambiguity
    stimuli

    ASJC Scopus subject areas

    • Acoustics and Ultrasonics
    • Arts and Humanities (miscellaneous)

    Cite this

    Visual capture of a stereo sound : Interactions between cue reliability, sound localization variability, and cross-modal bias. / Montagne, Christopher; Zhou, Yi.

    In: Journal of the Acoustical Society of America, Vol. 140, No. 1, 01.07.2016, p. 471-485.

    Research output: Contribution to journalArticle

    @article{173702178b1c4dd0878adf23c51a481e,
    title = "Visual capture of a stereo sound: Interactions between cue reliability, sound localization variability, and cross-modal bias",
    abstract = "Multisensory interactions involve coordination and sometimes competition between multiple senses. Vision usually dominates audition in spatial judgments when light and sound stimuli are presented from two different physical locations. This study investigated the influence of vision on the perceived location of a phantom sound source placed in a stereo sound field using a pair of loudspeakers emitting identical signals that were delayed or attenuated relative to each other. Results show that although a similar horizontal range (+/-45°) was reported for timing-modulated and level-modulated signals, listeners' localization performance showed greater variability for the timing signals. When visual stimuli were presented simultaneously with the auditory stimuli, listeners showed stronger visual bias for timing-modulated signals than level-modulated and single-speaker control signals. Trial-to-trial errors remained relatively stable over time, suggesting that sound localization uncertainty has an immediate and long-lasting effect on the across-modal bias. Binaural signal analyses further reveal that interaural differences of time and intensity - the two primary cues for sound localization in the azimuthal plane - are inherently more ambiguous for signals placed using timing. These results suggest that binaural ambiguity is intrinsically linked with localization variability and the strength of cross-modal bias in sound localization.",
    author = "Christopher Montagne and Yi Zhou",
    year = "2016",
    month = "7",
    day = "1",
    doi = "10.1121/1.4955314",
    language = "English (US)",
    volume = "140",
    pages = "471--485",
    journal = "Journal of the Acoustical Society of America",
    issn = "0001-4966",
    publisher = "Acoustical Society of America",
    number = "1",

    }

    TY - JOUR

    T1 - Visual capture of a stereo sound

    T2 - Interactions between cue reliability, sound localization variability, and cross-modal bias

    AU - Montagne, Christopher

    AU - Zhou, Yi

    PY - 2016/7/1

    Y1 - 2016/7/1

    N2 - Multisensory interactions involve coordination and sometimes competition between multiple senses. Vision usually dominates audition in spatial judgments when light and sound stimuli are presented from two different physical locations. This study investigated the influence of vision on the perceived location of a phantom sound source placed in a stereo sound field using a pair of loudspeakers emitting identical signals that were delayed or attenuated relative to each other. Results show that although a similar horizontal range (+/-45°) was reported for timing-modulated and level-modulated signals, listeners' localization performance showed greater variability for the timing signals. When visual stimuli were presented simultaneously with the auditory stimuli, listeners showed stronger visual bias for timing-modulated signals than level-modulated and single-speaker control signals. Trial-to-trial errors remained relatively stable over time, suggesting that sound localization uncertainty has an immediate and long-lasting effect on the across-modal bias. Binaural signal analyses further reveal that interaural differences of time and intensity - the two primary cues for sound localization in the azimuthal plane - are inherently more ambiguous for signals placed using timing. These results suggest that binaural ambiguity is intrinsically linked with localization variability and the strength of cross-modal bias in sound localization.

    AB - Multisensory interactions involve coordination and sometimes competition between multiple senses. Vision usually dominates audition in spatial judgments when light and sound stimuli are presented from two different physical locations. This study investigated the influence of vision on the perceived location of a phantom sound source placed in a stereo sound field using a pair of loudspeakers emitting identical signals that were delayed or attenuated relative to each other. Results show that although a similar horizontal range (+/-45°) was reported for timing-modulated and level-modulated signals, listeners' localization performance showed greater variability for the timing signals. When visual stimuli were presented simultaneously with the auditory stimuli, listeners showed stronger visual bias for timing-modulated signals than level-modulated and single-speaker control signals. Trial-to-trial errors remained relatively stable over time, suggesting that sound localization uncertainty has an immediate and long-lasting effect on the across-modal bias. Binaural signal analyses further reveal that interaural differences of time and intensity - the two primary cues for sound localization in the azimuthal plane - are inherently more ambiguous for signals placed using timing. These results suggest that binaural ambiguity is intrinsically linked with localization variability and the strength of cross-modal bias in sound localization.

    UR - http://www.scopus.com/inward/record.url?scp=84979201655&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=84979201655&partnerID=8YFLogxK

    U2 - 10.1121/1.4955314

    DO - 10.1121/1.4955314

    M3 - Article

    C2 - 27475171

    AN - SCOPUS:84979201655

    VL - 140

    SP - 471

    EP - 485

    JO - Journal of the Acoustical Society of America

    JF - Journal of the Acoustical Society of America

    SN - 0001-4966

    IS - 1

    ER -