Intrinsic-Motivated Sensor Management: Exploring with Physical Surprise

Jingyi Yuan, Yang Weng, Erik Blasch

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In modern complex physical systems, advanced sensing technologies extend the sensor coverage but also increase the difficulties of improving system monitoring capabilities based on real-time data availability. Traditional model-based methods of sensor management are limited to specific systems/settings, which can be challenged when system knowledge is intractable. Fortunately, the large amount of data collected in real-time allows machine learning methods to be a complement. Especially, reinforcement learning-based control is recognized for its capability to dynamically interact with systems. However, the direct implementation of learning methods easily overfits and results in inaccurate physics modeling for sensor management. Although physical regularization is a popular direction to bridge the gap, learning-based sensor control still suffers from convergence failure under highly complex and uncertain scenarios. This paper develops physics-embedded and self-supervised reinforcement learning for sensor management using an intrinsic reward. Specifically, the intrinsic-motivated sensor management (IMSM) constructs the local surprise information from the physical latent features, which captures hidden states in observations, and thus intrinsically motivates the agent to speed-up exploration. We show that the designs can not only relieve the lack of consistency with underlying physics/physical dynamics, but also adapt the global objective of maximizing monitoring capabilities to local environment changes. We demonstrate its effectiveness by experiments on physical system sensor control. The proposed model is implemented for the sensor management of unmanned vehicles and sensor rescheduling in complex/settled power systems, with or without observability constraints. Numerical results show that our model provides consistently higher threat detection accuracy and better observability recovery, as compared to existing methods.

Original languageEnglish (US)
Title of host publicationKDD 2022 - Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
PublisherAssociation for Computing Machinery
Pages2399-2407
Number of pages9
ISBN (Electronic)9781450393850
DOIs
StatePublished - Aug 14 2022
Event28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2022 - Washington, United States
Duration: Aug 14 2022Aug 18 2022

Publication series

NameProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Conference

Conference28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2022
Country/TerritoryUnited States
CityWashington
Period8/14/228/18/22

Keywords

  • intrinsic motivation
  • learning-based control
  • neural networks
  • power systems
  • reinforcement learning
  • sensor management
  • unmanned vehicles

ASJC Scopus subject areas

  • Software
  • Information Systems

Fingerprint

Dive into the research topics of 'Intrinsic-Motivated Sensor Management: Exploring with Physical Surprise'. Together they form a unique fingerprint.

Cite this