Correcting robot mistakes in real time using EEG signals

Andres F. Salazar-Gomez, Joseph Delpreto, Stephanie Gil, Frank H. Guenther, Daniela Rus

Research output: Chapter in Book/Report/Conference proceedingConference contribution

32 Citations (Scopus)

Abstract

Communication with a robot using brain activity from a human collaborator could provide a direct and fast feedback loop that is easy and natural for the human, thereby enabling a wide variety of intuitive interaction tasks. This paper explores the application of EEG-measured error-related potentials (ErrPs) to closed-loop robotic control. ErrP signals are particularly useful for robotics tasks because they are naturally occurring within the brain in response to an unexpected error. We decode ErrP signals from a human operator in real time to control a Rethink Robotics Baxter robot during a binary object selection task. We also show that utilizing a secondary interactive error-related potential signal generated during this closed-loop robot task can greatly improve classification performance, suggesting new ways in which robots can acquire human feedback. The design and implementation of the complete system is described, and results are presented for realtime closed-loop and open-loop experiments as well as offline analysis of both primary and secondary ErrP signals. These experiments are performed using general population subjects that have not been trained or screened. This work thereby demonstrates the potential for EEG-based feedback methods to facilitate seamless robotic control, and moves closer towards the goal of real-time intuitive interaction.

Original languageEnglish (US)
Title of host publicationICRA 2017 - IEEE International Conference on Robotics and Automation
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages6570-6577
Number of pages8
ISBN (Electronic)9781509046331
DOIs
StatePublished - Jul 21 2017
Externally publishedYes
Event2017 IEEE International Conference on Robotics and Automation, ICRA 2017 - Singapore, Singapore
Duration: May 29 2017Jun 3 2017

Other

Other2017 IEEE International Conference on Robotics and Automation, ICRA 2017
CountrySingapore
CitySingapore
Period5/29/176/3/17

Fingerprint

Electroencephalography
Robotics
Robots
Bioelectric potentials
Feedback
Brain
Experiments
Communication

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Artificial Intelligence
  • Electrical and Electronic Engineering

Cite this

Salazar-Gomez, A. F., Delpreto, J., Gil, S., Guenther, F. H., & Rus, D. (2017). Correcting robot mistakes in real time using EEG signals. In ICRA 2017 - IEEE International Conference on Robotics and Automation (pp. 6570-6577). [7989777] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICRA.2017.7989777

Correcting robot mistakes in real time using EEG signals. / Salazar-Gomez, Andres F.; Delpreto, Joseph; Gil, Stephanie; Guenther, Frank H.; Rus, Daniela.

ICRA 2017 - IEEE International Conference on Robotics and Automation. Institute of Electrical and Electronics Engineers Inc., 2017. p. 6570-6577 7989777.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Salazar-Gomez, AF, Delpreto, J, Gil, S, Guenther, FH & Rus, D 2017, Correcting robot mistakes in real time using EEG signals. in ICRA 2017 - IEEE International Conference on Robotics and Automation., 7989777, Institute of Electrical and Electronics Engineers Inc., pp. 6570-6577, 2017 IEEE International Conference on Robotics and Automation, ICRA 2017, Singapore, Singapore, 5/29/17. https://doi.org/10.1109/ICRA.2017.7989777
Salazar-Gomez AF, Delpreto J, Gil S, Guenther FH, Rus D. Correcting robot mistakes in real time using EEG signals. In ICRA 2017 - IEEE International Conference on Robotics and Automation. Institute of Electrical and Electronics Engineers Inc. 2017. p. 6570-6577. 7989777 https://doi.org/10.1109/ICRA.2017.7989777
Salazar-Gomez, Andres F. ; Delpreto, Joseph ; Gil, Stephanie ; Guenther, Frank H. ; Rus, Daniela. / Correcting robot mistakes in real time using EEG signals. ICRA 2017 - IEEE International Conference on Robotics and Automation. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 6570-6577
@inproceedings{b35484cb24434d00926af9b24f244150,
title = "Correcting robot mistakes in real time using EEG signals",
abstract = "Communication with a robot using brain activity from a human collaborator could provide a direct and fast feedback loop that is easy and natural for the human, thereby enabling a wide variety of intuitive interaction tasks. This paper explores the application of EEG-measured error-related potentials (ErrPs) to closed-loop robotic control. ErrP signals are particularly useful for robotics tasks because they are naturally occurring within the brain in response to an unexpected error. We decode ErrP signals from a human operator in real time to control a Rethink Robotics Baxter robot during a binary object selection task. We also show that utilizing a secondary interactive error-related potential signal generated during this closed-loop robot task can greatly improve classification performance, suggesting new ways in which robots can acquire human feedback. The design and implementation of the complete system is described, and results are presented for realtime closed-loop and open-loop experiments as well as offline analysis of both primary and secondary ErrP signals. These experiments are performed using general population subjects that have not been trained or screened. This work thereby demonstrates the potential for EEG-based feedback methods to facilitate seamless robotic control, and moves closer towards the goal of real-time intuitive interaction.",
author = "Salazar-Gomez, {Andres F.} and Joseph Delpreto and Stephanie Gil and Guenther, {Frank H.} and Daniela Rus",
year = "2017",
month = "7",
day = "21",
doi = "10.1109/ICRA.2017.7989777",
language = "English (US)",
pages = "6570--6577",
booktitle = "ICRA 2017 - IEEE International Conference on Robotics and Automation",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Correcting robot mistakes in real time using EEG signals

AU - Salazar-Gomez, Andres F.

AU - Delpreto, Joseph

AU - Gil, Stephanie

AU - Guenther, Frank H.

AU - Rus, Daniela

PY - 2017/7/21

Y1 - 2017/7/21

N2 - Communication with a robot using brain activity from a human collaborator could provide a direct and fast feedback loop that is easy and natural for the human, thereby enabling a wide variety of intuitive interaction tasks. This paper explores the application of EEG-measured error-related potentials (ErrPs) to closed-loop robotic control. ErrP signals are particularly useful for robotics tasks because they are naturally occurring within the brain in response to an unexpected error. We decode ErrP signals from a human operator in real time to control a Rethink Robotics Baxter robot during a binary object selection task. We also show that utilizing a secondary interactive error-related potential signal generated during this closed-loop robot task can greatly improve classification performance, suggesting new ways in which robots can acquire human feedback. The design and implementation of the complete system is described, and results are presented for realtime closed-loop and open-loop experiments as well as offline analysis of both primary and secondary ErrP signals. These experiments are performed using general population subjects that have not been trained or screened. This work thereby demonstrates the potential for EEG-based feedback methods to facilitate seamless robotic control, and moves closer towards the goal of real-time intuitive interaction.

AB - Communication with a robot using brain activity from a human collaborator could provide a direct and fast feedback loop that is easy and natural for the human, thereby enabling a wide variety of intuitive interaction tasks. This paper explores the application of EEG-measured error-related potentials (ErrPs) to closed-loop robotic control. ErrP signals are particularly useful for robotics tasks because they are naturally occurring within the brain in response to an unexpected error. We decode ErrP signals from a human operator in real time to control a Rethink Robotics Baxter robot during a binary object selection task. We also show that utilizing a secondary interactive error-related potential signal generated during this closed-loop robot task can greatly improve classification performance, suggesting new ways in which robots can acquire human feedback. The design and implementation of the complete system is described, and results are presented for realtime closed-loop and open-loop experiments as well as offline analysis of both primary and secondary ErrP signals. These experiments are performed using general population subjects that have not been trained or screened. This work thereby demonstrates the potential for EEG-based feedback methods to facilitate seamless robotic control, and moves closer towards the goal of real-time intuitive interaction.

UR - http://www.scopus.com/inward/record.url?scp=85028018231&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85028018231&partnerID=8YFLogxK

U2 - 10.1109/ICRA.2017.7989777

DO - 10.1109/ICRA.2017.7989777

M3 - Conference contribution

SP - 6570

EP - 6577

BT - ICRA 2017 - IEEE International Conference on Robotics and Automation

PB - Institute of Electrical and Electronics Engineers Inc.

ER -