Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams

Jeremy A. Marvel, Shelly Bagchi, Megan Zimmerman, Murat Aksu, Brian Antonishek, Yue Wang, Ross Mead, Terry Fong, Hani Ben Amor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many years, recent technological and scientific advancements show that, while their applications may differ, the underlying principles of HRI performance impact each identically. As such, this workshop seeks to identify test methods and metrics for the holistic assessment and assurance of collaborative HRI performance. The focus is on identifying the key performance indicators of these seemingly disparate sectors, and additionally to establish a community based on the principles of transparency, repeatability, establishing trust in the assessment of collaborative HRI. The goal is to aid in the advancement of HRI technologies through the development of experimental scenarios, protocols, test methods, metrics for the verification and validation of interaction solutions and interface designs.

Original languageEnglish (US)
Title of host publicationHRI 2019 - 14th ACM/IEEE International Conference on Human-Robot Interaction
PublisherIEEE Computer Society
Pages696-697
Number of pages2
ISBN (Electronic)9781538685556
DOIs
StatePublished - Mar 22 2019
Event14th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2019 - Daegu, Korea, Republic of
Duration: Mar 11 2019Mar 14 2019

Publication series

NameACM/IEEE International Conference on Human-Robot Interaction
Volume2019-March
ISSN (Electronic)2167-2148

Conference

Conference14th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2019
CountryKorea, Republic of
CityDaegu
Period3/11/193/14/19

Fingerprint

Human robot interaction
Robots
Benchmarking
Transparency
Large scale systems
Industry
Robotics
Innovation
Network protocols

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Electrical and Electronic Engineering

Cite this

Marvel, J. A., Bagchi, S., Zimmerman, M., Aksu, M., Antonishek, B., Wang, Y., ... Ben Amor, H. (2019). Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams. In HRI 2019 - 14th ACM/IEEE International Conference on Human-Robot Interaction (pp. 696-697). [8673149] (ACM/IEEE International Conference on Human-Robot Interaction; Vol. 2019-March). IEEE Computer Society. https://doi.org/10.1109/HRI.2019.8673149

Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams. / Marvel, Jeremy A.; Bagchi, Shelly; Zimmerman, Megan; Aksu, Murat; Antonishek, Brian; Wang, Yue; Mead, Ross; Fong, Terry; Ben Amor, Hani.

HRI 2019 - 14th ACM/IEEE International Conference on Human-Robot Interaction. IEEE Computer Society, 2019. p. 696-697 8673149 (ACM/IEEE International Conference on Human-Robot Interaction; Vol. 2019-March).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Marvel, JA, Bagchi, S, Zimmerman, M, Aksu, M, Antonishek, B, Wang, Y, Mead, R, Fong, T & Ben Amor, H 2019, Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams. in HRI 2019 - 14th ACM/IEEE International Conference on Human-Robot Interaction., 8673149, ACM/IEEE International Conference on Human-Robot Interaction, vol. 2019-March, IEEE Computer Society, pp. 696-697, 14th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2019, Daegu, Korea, Republic of, 3/11/19. https://doi.org/10.1109/HRI.2019.8673149
Marvel JA, Bagchi S, Zimmerman M, Aksu M, Antonishek B, Wang Y et al. Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams. In HRI 2019 - 14th ACM/IEEE International Conference on Human-Robot Interaction. IEEE Computer Society. 2019. p. 696-697. 8673149. (ACM/IEEE International Conference on Human-Robot Interaction). https://doi.org/10.1109/HRI.2019.8673149
Marvel, Jeremy A. ; Bagchi, Shelly ; Zimmerman, Megan ; Aksu, Murat ; Antonishek, Brian ; Wang, Yue ; Mead, Ross ; Fong, Terry ; Ben Amor, Hani. / Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams. HRI 2019 - 14th ACM/IEEE International Conference on Human-Robot Interaction. IEEE Computer Society, 2019. pp. 696-697 (ACM/IEEE International Conference on Human-Robot Interaction).
@inproceedings{a1f38276108c45b59df3914a06e300dd,
title = "Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams",
abstract = "Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many years, recent technological and scientific advancements show that, while their applications may differ, the underlying principles of HRI performance impact each identically. As such, this workshop seeks to identify test methods and metrics for the holistic assessment and assurance of collaborative HRI performance. The focus is on identifying the key performance indicators of these seemingly disparate sectors, and additionally to establish a community based on the principles of transparency, repeatability, establishing trust in the assessment of collaborative HRI. The goal is to aid in the advancement of HRI technologies through the development of experimental scenarios, protocols, test methods, metrics for the verification and validation of interaction solutions and interface designs.",
author = "Marvel, {Jeremy A.} and Shelly Bagchi and Megan Zimmerman and Murat Aksu and Brian Antonishek and Yue Wang and Ross Mead and Terry Fong and {Ben Amor}, Hani",
year = "2019",
month = "3",
day = "22",
doi = "10.1109/HRI.2019.8673149",
language = "English (US)",
series = "ACM/IEEE International Conference on Human-Robot Interaction",
publisher = "IEEE Computer Society",
pages = "696--697",
booktitle = "HRI 2019 - 14th ACM/IEEE International Conference on Human-Robot Interaction",

}

TY - GEN

T1 - Test Methods and Metrics for Effective HRI in Collaborative Human-Robot Teams

AU - Marvel, Jeremy A.

AU - Bagchi, Shelly

AU - Zimmerman, Megan

AU - Aksu, Murat

AU - Antonishek, Brian

AU - Wang, Yue

AU - Mead, Ross

AU - Fong, Terry

AU - Ben Amor, Hani

PY - 2019/3/22

Y1 - 2019/3/22

N2 - Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many years, recent technological and scientific advancements show that, while their applications may differ, the underlying principles of HRI performance impact each identically. As such, this workshop seeks to identify test methods and metrics for the holistic assessment and assurance of collaborative HRI performance. The focus is on identifying the key performance indicators of these seemingly disparate sectors, and additionally to establish a community based on the principles of transparency, repeatability, establishing trust in the assessment of collaborative HRI. The goal is to aid in the advancement of HRI technologies through the development of experimental scenarios, protocols, test methods, metrics for the verification and validation of interaction solutions and interface designs.

AB - Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many years, recent technological and scientific advancements show that, while their applications may differ, the underlying principles of HRI performance impact each identically. As such, this workshop seeks to identify test methods and metrics for the holistic assessment and assurance of collaborative HRI performance. The focus is on identifying the key performance indicators of these seemingly disparate sectors, and additionally to establish a community based on the principles of transparency, repeatability, establishing trust in the assessment of collaborative HRI. The goal is to aid in the advancement of HRI technologies through the development of experimental scenarios, protocols, test methods, metrics for the verification and validation of interaction solutions and interface designs.

UR - http://www.scopus.com/inward/record.url?scp=85063990424&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063990424&partnerID=8YFLogxK

U2 - 10.1109/HRI.2019.8673149

DO - 10.1109/HRI.2019.8673149

M3 - Conference contribution

T3 - ACM/IEEE International Conference on Human-Robot Interaction

SP - 696

EP - 697

BT - HRI 2019 - 14th ACM/IEEE International Conference on Human-Robot Interaction

PB - IEEE Computer Society

ER -