TY - GEN
T1 - Team situation awareness in human-autonomy teaming
T2 - 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018
AU - Grimm, David A.
AU - Demit, Mustafa
AU - Gorman, Jamie C.
AU - Cooke, Nancy J.
N1 - Funding Information:
This research was partially supported by ONR Award N000I41H0844 (Program Managers: Marc Steinberg, Paul Bello) and ONR Award N0001417I2382 (Program Managers: Marc Steinberg, Micah Clark). We also acknowledge the assistance of Steven M. Shope, Sandia Research Corporation who integrated the synthetic agent and the testbed.
Publisher Copyright:
© 2018 Human Factors an Ergonomics Society Inc.. All rights reserved.
PY - 2018
Y1 - 2018
N2 - Project overview. The current study focuses on analyzing team flexibility by measuring entropy (where higher values correspond to system reorganization and lower values correspond to more stable system organization) across all-human teams and Human-Autonomy Teams (HAT). We analyzed teams in the context of a fully-fledged synthetic agent that acts as a pilot for a three-agent Remotely Piloted Aircraft System (RPAS) ground crew. The synthetic agent must be able to communicate and coordinate with human teammates in a constructive and timely manner to be effective. This study involved three heterogeneous team members who had to take photographs of target waypoints and communicate via a text-based communication system. The three team members' roles were: 1) navigator provides information about flight plan with speed and altitude restrictions at each waypoint; 2) pilot adjusts altitude and airspeed to control the Remotely Piloted Aircraft (RPA), and negotiates with the photographer about the current altitude and airspeed to take good photos for the targets; and 3) photographer screens camera settings, and sends feedback to other team members regarding the target photograph status. The three conditions differed based on the manipulation of the pilot role: 1) Synthetic - the pilot was the synthetic agent, 2) Control - the pilot was a randomly assigned participant, and 3) Experimenter - the pilot was a well-trained experimenter who focused on sending and receiving information in a timely manner. The goal of this study is to examine how overall RPAS flexibility across HATs and all-human teams are associated with Team Situation Awareness (TSA). Method. There were 30 teams (10-teams per condition): control teams consisted of three participants randomly assigned to each role; synthetic and experimenter teams included two participants randomly assigned to the navigator and photographer roles. The experiment took place over five 40-minute missions, and the goal was to take as many "good" photos of ground targets as possible while avoiding alarms and rule violations. We obtained several measures, including mission and target level team performance scores, team process measures (situation awareness, process ratings, communication and coordination), and other measures (teamwork knowledge, workload, and demographics). We first estimated amount of system reorganization of the RPAS via an information entropy measure, i.e., the number of arrangements the system occupied over a given period of time (Shannon & Weaver, 1975). Based on information entropy, we defined four layers to represent the RPAS (Gorman, Demir, Cooke, & Grimm, In Review): 1) communications - the chat-based communication among team members; 2) vehicle - the RPA itself, e.g., speed, altitude; 3) control - interface between the RPA and the user; and system - the overall activity of the sub-layers. Then, we looked at the relationship between layers and TSA, which was based on successfully overcoming and completing ad hoc embedded target waypoints. Results and conclusion. Overall, the experimenter teams adapted to more roadblocks than the synthetic teams, who were equivalent to control teams (Demir, McNeese, & Cooke, 2016). The findings indicate that: I) synthetic teams demonstrated rigid systems level activity, which consisted of less reorganization of communication, control and vehicle layers as conditions changed, which also resulted in less adaptation to roadblocks; 2) control teams demonstrated less communication reorganization, but more control and vehicle reorganization, which also resulted in less adaptation to roadblocks; and 3) experimenter teams demonstrated more reorganization across communication, control and vehicle layers, which resulted in better adaptation to roadblocks. Thus, the ability of a system to reorganize across human and technical layers as situations change is needed to adapt to novel conditions of team performance in a dynamic task.
AB - Project overview. The current study focuses on analyzing team flexibility by measuring entropy (where higher values correspond to system reorganization and lower values correspond to more stable system organization) across all-human teams and Human-Autonomy Teams (HAT). We analyzed teams in the context of a fully-fledged synthetic agent that acts as a pilot for a three-agent Remotely Piloted Aircraft System (RPAS) ground crew. The synthetic agent must be able to communicate and coordinate with human teammates in a constructive and timely manner to be effective. This study involved three heterogeneous team members who had to take photographs of target waypoints and communicate via a text-based communication system. The three team members' roles were: 1) navigator provides information about flight plan with speed and altitude restrictions at each waypoint; 2) pilot adjusts altitude and airspeed to control the Remotely Piloted Aircraft (RPA), and negotiates with the photographer about the current altitude and airspeed to take good photos for the targets; and 3) photographer screens camera settings, and sends feedback to other team members regarding the target photograph status. The three conditions differed based on the manipulation of the pilot role: 1) Synthetic - the pilot was the synthetic agent, 2) Control - the pilot was a randomly assigned participant, and 3) Experimenter - the pilot was a well-trained experimenter who focused on sending and receiving information in a timely manner. The goal of this study is to examine how overall RPAS flexibility across HATs and all-human teams are associated with Team Situation Awareness (TSA). Method. There were 30 teams (10-teams per condition): control teams consisted of three participants randomly assigned to each role; synthetic and experimenter teams included two participants randomly assigned to the navigator and photographer roles. The experiment took place over five 40-minute missions, and the goal was to take as many "good" photos of ground targets as possible while avoiding alarms and rule violations. We obtained several measures, including mission and target level team performance scores, team process measures (situation awareness, process ratings, communication and coordination), and other measures (teamwork knowledge, workload, and demographics). We first estimated amount of system reorganization of the RPAS via an information entropy measure, i.e., the number of arrangements the system occupied over a given period of time (Shannon & Weaver, 1975). Based on information entropy, we defined four layers to represent the RPAS (Gorman, Demir, Cooke, & Grimm, In Review): 1) communications - the chat-based communication among team members; 2) vehicle - the RPA itself, e.g., speed, altitude; 3) control - interface between the RPA and the user; and system - the overall activity of the sub-layers. Then, we looked at the relationship between layers and TSA, which was based on successfully overcoming and completing ad hoc embedded target waypoints. Results and conclusion. Overall, the experimenter teams adapted to more roadblocks than the synthetic teams, who were equivalent to control teams (Demir, McNeese, & Cooke, 2016). The findings indicate that: I) synthetic teams demonstrated rigid systems level activity, which consisted of less reorganization of communication, control and vehicle layers as conditions changed, which also resulted in less adaptation to roadblocks; 2) control teams demonstrated less communication reorganization, but more control and vehicle reorganization, which also resulted in less adaptation to roadblocks; and 3) experimenter teams demonstrated more reorganization across communication, control and vehicle layers, which resulted in better adaptation to roadblocks. Thus, the ability of a system to reorganize across human and technical layers as situations change is needed to adapt to novel conditions of team performance in a dynamic task.
UR - http://www.scopus.com/inward/record.url?scp=85068213027&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85068213027&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85068213027
T3 - Proceedings of the Human Factors and Ergonomics Society
SP - 149
BT - 62nd Human Factors and Ergonomics Society Annual Meeting, HFES 2018
PB - Human Factors and Ergonomics Society Inc.
Y2 - 1 October 2018 through 5 October 2018
ER -