Robotic Knee Parameter Tuning Using Approximate Policy Iteration

Xiang Gao, Yue Wen, Minhan Li, Jennie Si, He (Helen) Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents an online model-free reinforcement learning based controller realized by approximate dynamic programming for a robotic knee as part of a human-machine system. Traditionally, prosthesis wearers’ gait performance is improved by manually tuning the impedance parameters. In this paper, we show that the parameter tuning problem can be formulated as an optimal control problem and thus solved by dynamic programming. Toward this goal, we constructed an quadratic instantaneous cost, which resulted in a value function that could be approximated by a neural network. The control policy is then solved by the least-squared method iteratively, a framework of which we refer to as approximate policy iteration. We performed extensive simulations based on prosthetic kinetics and human performance data extracted from real human subjects. Our results show that the proposed parameter tuning algorithm can be readily used for adaptive optimal tuning of prosthetic knee control parameters and the tuning process is time and sample efficient.

Original languageEnglish (US)
Title of host publicationCognitive Systems and Signal Processing - 4th International Conference, ICCSIP 2018, Revised Selected Papers
EditorsFuchun Sun, Huaping Liu, Dewen Hu
PublisherSpringer Verlag
Pages554-563
Number of pages10
ISBN (Print)9789811379826
DOIs
StatePublished - Jan 1 2019
Event4th International Conference on Cognitive Systems and Information Processing, ICCSIP 2018 - Beijing, China
Duration: Nov 29 2018Dec 1 2018

Publication series

NameCommunications in Computer and Information Science
Volume1005
ISSN (Print)1865-0929

Conference

Conference4th International Conference on Cognitive Systems and Information Processing, ICCSIP 2018
CountryChina
CityBeijing
Period11/29/1812/1/18

    Fingerprint

Keywords

  • Approximate dynamic programming (ADP)
  • Lower limb prosthesis
  • Policy iteration
  • Sample efficient learning

ASJC Scopus subject areas

  • Computer Science(all)
  • Mathematics(all)

Cite this

Gao, X., Wen, Y., Li, M., Si, J., & Huang, H. H. (2019). Robotic Knee Parameter Tuning Using Approximate Policy Iteration. In F. Sun, H. Liu, & D. Hu (Eds.), Cognitive Systems and Signal Processing - 4th International Conference, ICCSIP 2018, Revised Selected Papers (pp. 554-563). (Communications in Computer and Information Science; Vol. 1005). Springer Verlag. https://doi.org/10.1007/978-981-13-7983-3_49