Perception and action interfaces in the symbiosis of humans and multi-agent systems

Background According to the recently published U.S. Air Force Global Science and Technology Vision report, there is growing interest in many government agencies to design and build networks of unmanned vehicles that will have the ability of operating autonomously [1]. Regarding unmanned aerial vehicles (UAVs), present day systems have diminished the human role to supervisory control in order to include higher level cognitive reasoning. Making the transition from a single UAV to a swarm of multiple ones is not straightforward though. How humans can supervise and control a swarm of artificial semi-autonomous systems has not been adequately researched. In fact, the research on mixed human-machine systems to aid inference, communication, planning and decision making, in the case where the machine is composed of many independent semi-autonomous systems, is still at its infancy. Figure 1: Transition from controlling one to many agents. State-of-the-art systems involve usually one human controller for a single UAV. The human has spatial feedback of the controlled vehicle, and provides it with high-level commands (e.g. fly to a specific location or follow predefined surveillance path) [2, 3]. However, the swarming paradigm, deriving inspiration from the behavior of natural swarms such as bird flocks and fish schools, offers myriad advantages to a team of UAVs. A swarm system consists of a large group of relatively inexpensive, interchangeable vehicles that execute autonomous decisions using information obtained via local sensing and communication. The redundancy in a swarm makes its operation robust to vehicle failures and disturbances, which also enables the use of sacrificial platforms, and its distributed activity can conceal the systems mission from an opponent. Recent advances in computing, sensing, actuation and control technologies are currently enabling the development of swarms of aerial vehicles, varying in complexity, size and overall scale [4, 5]. As the power of the many UAVs is facilitating an increasing number of military missions needs, the human role in the high-level control architecture of this population is getting more significant. Humans are capable of understanding high-level goals of a group of agents, e.g. a colony of ants, without focusing on each ant individually. However, the perception-decision-action paradigm in humans, when it involves behavioral data of a group of multiple actors instead of one, is not well understood yet. More specifically, the principles controlling the way humans perceive information and make decisions, when the number of observed actors is becoming more than one as illustrated in Fig. 1, remains undiscovered.