TY - JOUR
T1 - Dancing With Algorithms
T2 - Interaction Creates Greater Preference and Trust in Machine-Learned Behavior
AU - Gutzwiller, Robert S.
AU - Reeder, John
N1 - Funding Information:
This project was partially supported by the Office of the Secretary of Defense (OSD) Autonomous Research Pilot Initiative (ARPI) project to SPAWAR (now Naval Information Warfare Center) Pacific and by an internal grant from the Office and Naval Research. The current work supersedes any of the prior claims and results reported in Gutzwiller and Reeder (2017), and no further citations of that report should be made.
Publisher Copyright:
© Copyright 2020, Human Factors and Ergonomics Society.
PY - 2021/8
Y1 - 2021/8
N2 - Objective: We examined a method of machine learning (ML) to evaluate its potential to develop more trustworthy control of unmanned vehicle area search behaviors. Background: ML typically lacks interaction with the user. Novel interactive machine learning (IML) techniques incorporate user feedback, enabling observation of emerging ML behaviors, and human collaboration during ML of a task. This may enable trust and recognition of these algorithms. Method: Participants judged and selected behaviors in a low and a high interaction condition (IML) over the course of behavior evolution using ML. User trust in the outputs, as well as preference, and ability to discriminate and recognize the behaviors were measured. Results: Compared to noninteractive techniques, IML behaviors were more trusted and preferred, as well as recognizable, separate from non-IML behaviors, and approached similar performance as pure ML models. Conclusion: IML shows promise for creating behaviors by involving the user; this is the first extension of this technique for vehicle behavior model development targeting user satisfaction and is unique in its multifaceted evaluation of how users perceived, trusted, and implemented these learned controllers. Application: There are many contexts where the brittleness of ML cannot be trusted, but the advantage of ML over traditional programmed behaviors may be large, as in some military operations where they could be scaled. IML in this early form appears to generate satisfactory behaviors without sacrificing performance, use, or trust in the behavior, but more work is necessary.
AB - Objective: We examined a method of machine learning (ML) to evaluate its potential to develop more trustworthy control of unmanned vehicle area search behaviors. Background: ML typically lacks interaction with the user. Novel interactive machine learning (IML) techniques incorporate user feedback, enabling observation of emerging ML behaviors, and human collaboration during ML of a task. This may enable trust and recognition of these algorithms. Method: Participants judged and selected behaviors in a low and a high interaction condition (IML) over the course of behavior evolution using ML. User trust in the outputs, as well as preference, and ability to discriminate and recognize the behaviors were measured. Results: Compared to noninteractive techniques, IML behaviors were more trusted and preferred, as well as recognizable, separate from non-IML behaviors, and approached similar performance as pure ML models. Conclusion: IML shows promise for creating behaviors by involving the user; this is the first extension of this technique for vehicle behavior model development targeting user satisfaction and is unique in its multifaceted evaluation of how users perceived, trusted, and implemented these learned controllers. Application: There are many contexts where the brittleness of ML cannot be trusted, but the advantage of ML over traditional programmed behaviors may be large, as in some military operations where they could be scaled. IML in this early form appears to generate satisfactory behaviors without sacrificing performance, use, or trust in the behavior, but more work is necessary.
KW - automated agents
KW - human–automation interaction
KW - human–systems integration
KW - machine learning
KW - trust in automation
KW - uninhabited aerial vehicles
UR - http://www.scopus.com/inward/record.url?scp=85079403546&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85079403546&partnerID=8YFLogxK
U2 - 10.1177/0018720820903893
DO - 10.1177/0018720820903893
M3 - Article
C2 - 32048883
AN - SCOPUS:85079403546
VL - 63
SP - 854
EP - 867
JO - Human Factors
JF - Human Factors
SN - 0018-7208
IS - 5
ER -