What is it you really want of me? Generalized reward learning with biased beliefs about domain dynamics

Ze Gong, Yu Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reward learning as a method for inferring human intent and preferences has been studied extensively. Prior approaches make an implicit assumption that the human maintains a correct belief about the robot’s domain dynamics. However, this may not always hold since the human’s belief may be biased, which can ultimately lead to a misguided estimation of the human’s intent and preferences, which is often derived from human feedback on the robot’s behaviors. In this paper, we remove this restrictive assumption by considering that the human may have an inaccurate understanding of the robot. We propose a method called Generalized Reward Learning with biased beliefs about domain dynamics (GeReL) to infer both the reward function and human’s belief about the robot in a Bayesian setting based on human ratings. Due to the complex forms of the posteriors, we formulate it as a variational inference problem to infer the posteriors of the parameters that govern the reward function and human’s belief about the robot simultaneously. We evaluate our method in a simulated domain and with a user study where the user has a bias based on the robot’s appearances. The results show that our method can recover the true human preferences while subject to such biased beliefs, in contrast to prior approaches that could have misinterpreted them completely.

Original languageEnglish (US)
Title of host publicationAAAI 2020 - 34th AAAI Conference on Artificial Intelligence
PublisherAAAI press
Pages2485-2492
Number of pages8
ISBN (Electronic)9781577358350
StatePublished - 2020
Event34th AAAI Conference on Artificial Intelligence, AAAI 2020 - New York, United States
Duration: Feb 7 2020Feb 12 2020

Publication series

NameAAAI 2020 - 34th AAAI Conference on Artificial Intelligence

Conference

Conference34th AAAI Conference on Artificial Intelligence, AAAI 2020
Country/TerritoryUnited States
CityNew York
Period2/7/202/12/20

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'What is it you really want of me? Generalized reward learning with biased beliefs about domain dynamics'. Together they form a unique fingerprint.

Cite this