Sparse latent space policy search

Kevin Sebastian Luck, Joni Pajarinen, Erik Berger, Ville Kyrki, Hani Ben Amor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sampleefficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method uses Variational Inference to estimate policy parameters, while at the same time uncovering a lowdimensional latent space of controls. Prior knowledge about the task and the structure of the learning agent can be provided by specifying groups of potentially correlated parameters. This information is then used to impose sparsity constraints on the mapping between the high-dimensional space of controls and a lowerdimensional latent space. In experiments with a simulated bi-manual manipulator, the new approach effectively identifies synergies between joints, performs efficient low-dimensional policy search, and outperforms state-of-the-art policy search methods.

Original languageEnglish (US)
Title of host publication30th AAAI Conference on Artificial Intelligence, AAAI 2016
PublisherAAAI press
Pages1911-1918
Number of pages8
ISBN (Electronic)9781577357605
StatePublished - 2016
Event30th AAAI Conference on Artificial Intelligence, AAAI 2016 - Phoenix, United States
Duration: Feb 12 2016Feb 17 2016

Other

Other30th AAAI Conference on Artificial Intelligence, AAAI 2016
CountryUnited States
CityPhoenix
Period2/12/162/17/16

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Sparse latent space policy search'. Together they form a unique fingerprint.

  • Cite this

    Luck, K. S., Pajarinen, J., Berger, E., Kyrki, V., & Ben Amor, H. (2016). Sparse latent space policy search. In 30th AAAI Conference on Artificial Intelligence, AAAI 2016 (pp. 1911-1918). AAAI press.