Directing policy search with interactively taught via-points

Yannick Schroecker, Hani Ben Amor, Andrea Thomaz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Policy search has been successfully applied to robot motor learning problems. However, for moderately complex tasks the necessity of good heuristics or initialization still arises. One method that has been used to alleviate this problem is to utilize demonstrations obtained by a human teacher as a starting point for policy search in the space of trajectories. In this paper we describe an alternative way of giving demonstrations as soft via-points and show how they can be used for initialization as well as for active corrections during the learning process. With this approach, we restrict the search space to trajectories that will be close to the taught via-points at the taught time and thereby significantly reduce the number of samples necessary to learn a good policy. We show with a simulated robot arm that our method can efficiently learn to insert an object in a hole with just a minimal demonstration and evaluate our method further on a synthetic letter reproduction task.

Original languageEnglish (US)
Title of host publicationAAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Pages1052-1059
Number of pages8
ISBN (Electronic)9781450342391
StatePublished - 2016
Event15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016 - Singapore, Singapore
Duration: May 9 2016May 13 2016

Other

Other15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016
CountrySingapore
CitySingapore
Period5/9/165/13/16

Fingerprint

Demonstrations
Trajectories
Robots

Keywords

  • Dynamic movement primitives
  • Keyframe demonstrations
  • Learning from demonstration
  • Reinforcement learning
  • Reinforcement learning for motor skills

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering

Cite this

Schroecker, Y., Ben Amor, H., & Thomaz, A. (2016). Directing policy search with interactively taught via-points. In AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems (pp. 1052-1059). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).

Directing policy search with interactively taught via-points. / Schroecker, Yannick; Ben Amor, Hani; Thomaz, Andrea.

AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2016. p. 1052-1059.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Schroecker, Y, Ben Amor, H & Thomaz, A 2016, Directing policy search with interactively taught via-points. in AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), pp. 1052-1059, 15th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2016, Singapore, Singapore, 5/9/16.
Schroecker Y, Ben Amor H, Thomaz A. Directing policy search with interactively taught via-points. In AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). 2016. p. 1052-1059
Schroecker, Yannick ; Ben Amor, Hani ; Thomaz, Andrea. / Directing policy search with interactively taught via-points. AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2016. pp. 1052-1059
@inproceedings{6ca809ff5cab469d88e034a6f16ea436,
title = "Directing policy search with interactively taught via-points",
abstract = "Policy search has been successfully applied to robot motor learning problems. However, for moderately complex tasks the necessity of good heuristics or initialization still arises. One method that has been used to alleviate this problem is to utilize demonstrations obtained by a human teacher as a starting point for policy search in the space of trajectories. In this paper we describe an alternative way of giving demonstrations as soft via-points and show how they can be used for initialization as well as for active corrections during the learning process. With this approach, we restrict the search space to trajectories that will be close to the taught via-points at the taught time and thereby significantly reduce the number of samples necessary to learn a good policy. We show with a simulated robot arm that our method can efficiently learn to insert an object in a hole with just a minimal demonstration and evaluate our method further on a synthetic letter reproduction task.",
keywords = "Dynamic movement primitives, Keyframe demonstrations, Learning from demonstration, Reinforcement learning, Reinforcement learning for motor skills",
author = "Yannick Schroecker and {Ben Amor}, Hani and Andrea Thomaz",
year = "2016",
language = "English (US)",
pages = "1052--1059",
booktitle = "AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems",
publisher = "International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)",

}

TY - GEN

T1 - Directing policy search with interactively taught via-points

AU - Schroecker, Yannick

AU - Ben Amor, Hani

AU - Thomaz, Andrea

PY - 2016

Y1 - 2016

N2 - Policy search has been successfully applied to robot motor learning problems. However, for moderately complex tasks the necessity of good heuristics or initialization still arises. One method that has been used to alleviate this problem is to utilize demonstrations obtained by a human teacher as a starting point for policy search in the space of trajectories. In this paper we describe an alternative way of giving demonstrations as soft via-points and show how they can be used for initialization as well as for active corrections during the learning process. With this approach, we restrict the search space to trajectories that will be close to the taught via-points at the taught time and thereby significantly reduce the number of samples necessary to learn a good policy. We show with a simulated robot arm that our method can efficiently learn to insert an object in a hole with just a minimal demonstration and evaluate our method further on a synthetic letter reproduction task.

AB - Policy search has been successfully applied to robot motor learning problems. However, for moderately complex tasks the necessity of good heuristics or initialization still arises. One method that has been used to alleviate this problem is to utilize demonstrations obtained by a human teacher as a starting point for policy search in the space of trajectories. In this paper we describe an alternative way of giving demonstrations as soft via-points and show how they can be used for initialization as well as for active corrections during the learning process. With this approach, we restrict the search space to trajectories that will be close to the taught via-points at the taught time and thereby significantly reduce the number of samples necessary to learn a good policy. We show with a simulated robot arm that our method can efficiently learn to insert an object in a hole with just a minimal demonstration and evaluate our method further on a synthetic letter reproduction task.

KW - Dynamic movement primitives

KW - Keyframe demonstrations

KW - Learning from demonstration

KW - Reinforcement learning

KW - Reinforcement learning for motor skills

UR - http://www.scopus.com/inward/record.url?scp=85014153645&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014153645&partnerID=8YFLogxK

M3 - Conference contribution

SP - 1052

EP - 1059

BT - AAMAS 2016 - Proceedings of the 2016 International Conference on Autonomous Agents and Multiagent Systems

PB - International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)

ER -