Why? Why not? When? Visual Explanations of Agent Behaviour in Reinforcement Learning

Aditi Mishra, Utkarsh Soni, Jinbin Huang, Chris Bryan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reinforcement learning (RL) is used in many domains, including autonomous driving, robotics, stock trading, and video games. Unfortunately, the black box nature of RL agents, combined with legal and ethical considerations, makes it increasingly important that humans (including those are who not experts in RL) understand the reasoning behind the actions taken by an RL agent, particularly in safety-critical domains. To help address this challenge, we introduce PolicyExplainer, a visual analytics interface which lets the user directly query an autonomous agent. PolicyExplainer visualizes the states, policy, and expected future rewards for an agent, and supports asking and answering questions such as: 'Why take this action? Why not take this other action? When is this action taken?' PolicyExplainer is designed based upon a domain analysis with RL researchers, and is evaluated via qualitative and quantitative assessments on a trio of domains: taxi navigation, a stack bot domain, and drug recommendation for HIV patients. We find that PolicyExplainer's visual approach promotes trust and understanding of agent decisions better than a state-of-the-art text-based explanation approach. Interviews with domain practitioners provide further validation for PolicyExplainer as applied to safety-critical domains. Our results help demonstrate how visualization-based approaches can be leveraged to decode the behavior of autonomous RL agents, particularly for RL non-experts.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 IEEE 15th Pacific Visualization Symposium, PacificVis 2022
PublisherIEEE Computer Society
Pages111-120
Number of pages10
ISBN (Electronic)9781665423359
DOIs
StatePublished - 2022
Externally publishedYes
Event15th IEEE Pacific Visualization Symposium, PacificVis 2022 - Virtual, Online, Japan
Duration: Apr 11 2022Apr 14 2022

Publication series

NameIEEE Pacific Visualization Symposium
Volume2022-April
ISSN (Print)2165-8765
ISSN (Electronic)2165-8773

Conference

Conference15th IEEE Pacific Visualization Symposium, PacificVis 2022
Country/TerritoryJapan
CityVirtual, Online
Period4/11/224/14/22

Keywords

  • Human-centered computing-Visualization-Visualization design and evaluation methods
  • Human-centered computing-Visualization-Visualization techniques-Treemaps

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Hardware and Architecture
  • Software

Fingerprint

Dive into the research topics of 'Why? Why not? When? Visual Explanations of Agent Behaviour in Reinforcement Learning'. Together they form a unique fingerprint.

Cite this