Sparse latent space policy search

Kevin Sebastian Luck, Joni Pajarinen, Erik Berger, Ville Kyrki, Hani Ben Amor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sampleefficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method uses Variational Inference to estimate policy parameters, while at the same time uncovering a lowdimensional latent space of controls. Prior knowledge about the task and the structure of the learning agent can be provided by specifying groups of potentially correlated parameters. This information is then used to impose sparsity constraints on the mapping between the high-dimensional space of controls and a lowerdimensional latent space. In experiments with a simulated bi-manual manipulator, the new approach effectively identifies synergies between joints, performs efficient low-dimensional policy search, and outperforms state-of-the-art policy search methods.

Original languageEnglish (US)
Title of host publication30th AAAI Conference on Artificial Intelligence, AAAI 2016
PublisherAAAI press
Pages1911-1918
Number of pages8
ISBN (Electronic)9781577357605
StatePublished - 2016
Event30th AAAI Conference on Artificial Intelligence, AAAI 2016 - Phoenix, United States
Duration: Feb 12 2016Feb 17 2016

Other

Other30th AAAI Conference on Artificial Intelligence, AAAI 2016
CountryUnited States
CityPhoenix
Period2/12/162/17/16

Fingerprint

Reinforcement learning
Manipulators
Robots
Experiments

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Luck, K. S., Pajarinen, J., Berger, E., Kyrki, V., & Ben Amor, H. (2016). Sparse latent space policy search. In 30th AAAI Conference on Artificial Intelligence, AAAI 2016 (pp. 1911-1918). AAAI press.

Sparse latent space policy search. / Luck, Kevin Sebastian; Pajarinen, Joni; Berger, Erik; Kyrki, Ville; Ben Amor, Hani.

30th AAAI Conference on Artificial Intelligence, AAAI 2016. AAAI press, 2016. p. 1911-1918.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Luck, KS, Pajarinen, J, Berger, E, Kyrki, V & Ben Amor, H 2016, Sparse latent space policy search. in 30th AAAI Conference on Artificial Intelligence, AAAI 2016. AAAI press, pp. 1911-1918, 30th AAAI Conference on Artificial Intelligence, AAAI 2016, Phoenix, United States, 2/12/16.
Luck KS, Pajarinen J, Berger E, Kyrki V, Ben Amor H. Sparse latent space policy search. In 30th AAAI Conference on Artificial Intelligence, AAAI 2016. AAAI press. 2016. p. 1911-1918
Luck, Kevin Sebastian ; Pajarinen, Joni ; Berger, Erik ; Kyrki, Ville ; Ben Amor, Hani. / Sparse latent space policy search. 30th AAAI Conference on Artificial Intelligence, AAAI 2016. AAAI press, 2016. pp. 1911-1918
@inproceedings{044d66d8b7d341b28360d1e5a4c8d031,
title = "Sparse latent space policy search",
abstract = "Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sampleefficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method uses Variational Inference to estimate policy parameters, while at the same time uncovering a lowdimensional latent space of controls. Prior knowledge about the task and the structure of the learning agent can be provided by specifying groups of potentially correlated parameters. This information is then used to impose sparsity constraints on the mapping between the high-dimensional space of controls and a lowerdimensional latent space. In experiments with a simulated bi-manual manipulator, the new approach effectively identifies synergies between joints, performs efficient low-dimensional policy search, and outperforms state-of-the-art policy search methods.",
author = "Luck, {Kevin Sebastian} and Joni Pajarinen and Erik Berger and Ville Kyrki and {Ben Amor}, Hani",
year = "2016",
language = "English (US)",
pages = "1911--1918",
booktitle = "30th AAAI Conference on Artificial Intelligence, AAAI 2016",
publisher = "AAAI press",

}

TY - GEN

T1 - Sparse latent space policy search

AU - Luck, Kevin Sebastian

AU - Pajarinen, Joni

AU - Berger, Erik

AU - Kyrki, Ville

AU - Ben Amor, Hani

PY - 2016

Y1 - 2016

N2 - Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sampleefficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method uses Variational Inference to estimate policy parameters, while at the same time uncovering a lowdimensional latent space of controls. Prior knowledge about the task and the structure of the learning agent can be provided by specifying groups of potentially correlated parameters. This information is then used to impose sparsity constraints on the mapping between the high-dimensional space of controls and a lowerdimensional latent space. In experiments with a simulated bi-manual manipulator, the new approach effectively identifies synergies between joints, performs efficient low-dimensional policy search, and outperforms state-of-the-art policy search methods.

AB - Computational agents often need to learn policies that involve many control variables, e.g., a robot needs to control several joints simultaneously. Learning a policy with a high number of parameters, however, usually requires a large number of training samples. We introduce a reinforcement learning method for sampleefficient policy search that exploits correlations between control variables. Such correlations are particularly frequent in motor skill learning tasks. The introduced method uses Variational Inference to estimate policy parameters, while at the same time uncovering a lowdimensional latent space of controls. Prior knowledge about the task and the structure of the learning agent can be provided by specifying groups of potentially correlated parameters. This information is then used to impose sparsity constraints on the mapping between the high-dimensional space of controls and a lowerdimensional latent space. In experiments with a simulated bi-manual manipulator, the new approach effectively identifies synergies between joints, performs efficient low-dimensional policy search, and outperforms state-of-the-art policy search methods.

UR - http://www.scopus.com/inward/record.url?scp=85007210846&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85007210846&partnerID=8YFLogxK

M3 - Conference contribution

SP - 1911

EP - 1918

BT - 30th AAAI Conference on Artificial Intelligence, AAAI 2016

PB - AAAI press

ER -