Do multimodal signals need to come from the same place? Crossmodal attentional links between proximal and distal surfaces

Robert Gray, H. Z. Tan, J. J. Young

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

Previous research has shown that the use of multimodal signals can lead to faster and more accurate responses compared to purely unimodal displays. However, in most cases response facilitation only occurs when the signals are presented in roughly the same spatial location. This would suggest a severe restriction on interface designers: to use multimodal displays effectively all signals must be presented from the same location on the display. We previously reported evidence that the use of haptic cues may provide a solution to this problem as haptic cues presented to a user's back can be used to redirect visual attention to locations on a screen in front of the user (Tan et al., 2001). In the present experiment we used a visual change detection task to investigate whether (i) this type of visual-haptic interaction is robust at low cue validity rates and (ii) similar effects occur for auditory cues. Valid haptic cues resulted in significantly faster change detection times even when they accurately indicated the location of the change on only 20% of the trials. Auditory cues had a much smaller effect on detection times at the high validity rate (80%) than haptic cues and did not significantly improve performance at the 20% validity rate. These results suggest that the use haptic attentional cues may be particularly effective in environments in which information cannot be presented in the same spatial location.

Original languageEnglish (US)
Title of host publicationProceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages437-441
Number of pages5
ISBN (Print)0769518346, 9780769518343
DOIs
StatePublished - 2002
Event4th IEEE International Conference on Multimodal Interfaces, ICMI 2002 - Pittsburgh, United States
Duration: Oct 14 2002Oct 16 2002

Other

Other4th IEEE International Conference on Multimodal Interfaces, ICMI 2002
CountryUnited States
CityPittsburgh
Period10/14/0210/16/02

Fingerprint

Display devices
Experiments

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Hardware and Architecture

Cite this

Gray, R., Tan, H. Z., & Young, J. J. (2002). Do multimodal signals need to come from the same place? Crossmodal attentional links between proximal and distal surfaces. In Proceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002 (pp. 437-441). [1167035] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMI.2002.1167035

Do multimodal signals need to come from the same place? Crossmodal attentional links between proximal and distal surfaces. / Gray, Robert; Tan, H. Z.; Young, J. J.

Proceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002. Institute of Electrical and Electronics Engineers Inc., 2002. p. 437-441 1167035.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Gray, R, Tan, HZ & Young, JJ 2002, Do multimodal signals need to come from the same place? Crossmodal attentional links between proximal and distal surfaces. in Proceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002., 1167035, Institute of Electrical and Electronics Engineers Inc., pp. 437-441, 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002, Pittsburgh, United States, 10/14/02. https://doi.org/10.1109/ICMI.2002.1167035
Gray R, Tan HZ, Young JJ. Do multimodal signals need to come from the same place? Crossmodal attentional links between proximal and distal surfaces. In Proceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002. Institute of Electrical and Electronics Engineers Inc. 2002. p. 437-441. 1167035 https://doi.org/10.1109/ICMI.2002.1167035
Gray, Robert ; Tan, H. Z. ; Young, J. J. / Do multimodal signals need to come from the same place? Crossmodal attentional links between proximal and distal surfaces. Proceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002. Institute of Electrical and Electronics Engineers Inc., 2002. pp. 437-441
@inproceedings{4b1e2191d1a147d6980fad81fd909034,
title = "Do multimodal signals need to come from the same place? Crossmodal attentional links between proximal and distal surfaces",
abstract = "Previous research has shown that the use of multimodal signals can lead to faster and more accurate responses compared to purely unimodal displays. However, in most cases response facilitation only occurs when the signals are presented in roughly the same spatial location. This would suggest a severe restriction on interface designers: to use multimodal displays effectively all signals must be presented from the same location on the display. We previously reported evidence that the use of haptic cues may provide a solution to this problem as haptic cues presented to a user's back can be used to redirect visual attention to locations on a screen in front of the user (Tan et al., 2001). In the present experiment we used a visual change detection task to investigate whether (i) this type of visual-haptic interaction is robust at low cue validity rates and (ii) similar effects occur for auditory cues. Valid haptic cues resulted in significantly faster change detection times even when they accurately indicated the location of the change on only 20{\%} of the trials. Auditory cues had a much smaller effect on detection times at the high validity rate (80{\%}) than haptic cues and did not significantly improve performance at the 20{\%} validity rate. These results suggest that the use haptic attentional cues may be particularly effective in environments in which information cannot be presented in the same spatial location.",
author = "Robert Gray and Tan, {H. Z.} and Young, {J. J.}",
year = "2002",
doi = "10.1109/ICMI.2002.1167035",
language = "English (US)",
isbn = "0769518346",
pages = "437--441",
booktitle = "Proceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Do multimodal signals need to come from the same place? Crossmodal attentional links between proximal and distal surfaces

AU - Gray, Robert

AU - Tan, H. Z.

AU - Young, J. J.

PY - 2002

Y1 - 2002

N2 - Previous research has shown that the use of multimodal signals can lead to faster and more accurate responses compared to purely unimodal displays. However, in most cases response facilitation only occurs when the signals are presented in roughly the same spatial location. This would suggest a severe restriction on interface designers: to use multimodal displays effectively all signals must be presented from the same location on the display. We previously reported evidence that the use of haptic cues may provide a solution to this problem as haptic cues presented to a user's back can be used to redirect visual attention to locations on a screen in front of the user (Tan et al., 2001). In the present experiment we used a visual change detection task to investigate whether (i) this type of visual-haptic interaction is robust at low cue validity rates and (ii) similar effects occur for auditory cues. Valid haptic cues resulted in significantly faster change detection times even when they accurately indicated the location of the change on only 20% of the trials. Auditory cues had a much smaller effect on detection times at the high validity rate (80%) than haptic cues and did not significantly improve performance at the 20% validity rate. These results suggest that the use haptic attentional cues may be particularly effective in environments in which information cannot be presented in the same spatial location.

AB - Previous research has shown that the use of multimodal signals can lead to faster and more accurate responses compared to purely unimodal displays. However, in most cases response facilitation only occurs when the signals are presented in roughly the same spatial location. This would suggest a severe restriction on interface designers: to use multimodal displays effectively all signals must be presented from the same location on the display. We previously reported evidence that the use of haptic cues may provide a solution to this problem as haptic cues presented to a user's back can be used to redirect visual attention to locations on a screen in front of the user (Tan et al., 2001). In the present experiment we used a visual change detection task to investigate whether (i) this type of visual-haptic interaction is robust at low cue validity rates and (ii) similar effects occur for auditory cues. Valid haptic cues resulted in significantly faster change detection times even when they accurately indicated the location of the change on only 20% of the trials. Auditory cues had a much smaller effect on detection times at the high validity rate (80%) than haptic cues and did not significantly improve performance at the 20% validity rate. These results suggest that the use haptic attentional cues may be particularly effective in environments in which information cannot be presented in the same spatial location.

UR - http://www.scopus.com/inward/record.url?scp=15744378494&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=15744378494&partnerID=8YFLogxK

U2 - 10.1109/ICMI.2002.1167035

DO - 10.1109/ICMI.2002.1167035

M3 - Conference contribution

SN - 0769518346

SN - 9780769518343

SP - 437

EP - 441

BT - Proceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002

PB - Institute of Electrical and Electronics Engineers Inc.

ER -