Abstract
This paper presents an online model-free reinforcement learning based controller realized by approximate dynamic programming for a robotic knee as part of a human-machine system. Traditionally, prosthesis wearers’ gait performance is improved by manually tuning the impedance parameters. In this paper, we show that the parameter tuning problem can be formulated as an optimal control problem and thus solved by dynamic programming. Toward this goal, we constructed an quadratic instantaneous cost, which resulted in a value function that could be approximated by a neural network. The control policy is then solved by the least-squared method iteratively, a framework of which we refer to as approximate policy iteration. We performed extensive simulations based on prosthetic kinetics and human performance data extracted from real human subjects. Our results show that the proposed parameter tuning algorithm can be readily used for adaptive optimal tuning of prosthetic knee control parameters and the tuning process is time and sample efficient.
| Original language | English (US) |
|---|---|
| Title of host publication | Cognitive Systems and Signal Processing - 4th International Conference, ICCSIP 2018, Revised Selected Papers |
| Editors | Fuchun Sun, Huaping Liu, Dewen Hu |
| Publisher | Springer Verlag |
| Pages | 554-563 |
| Number of pages | 10 |
| ISBN (Print) | 9789811379826 |
| DOIs | |
| State | Published - Jan 1 2019 |
| Event | 4th International Conference on Cognitive Systems and Information Processing, ICCSIP 2018 - Beijing, China Duration: Nov 29 2018 → Dec 1 2018 |
Publication series
| Name | Communications in Computer and Information Science |
|---|---|
| Volume | 1005 |
| ISSN (Print) | 1865-0929 |
Conference
| Conference | 4th International Conference on Cognitive Systems and Information Processing, ICCSIP 2018 |
|---|---|
| Country | China |
| City | Beijing |
| Period | 11/29/18 → 12/1/18 |
Fingerprint
Keywords
- Approximate dynamic programming (ADP)
- Lower limb prosthesis
- Policy iteration
- Sample efficient learning
ASJC Scopus subject areas
- Computer Science(all)
- Mathematics(all)
Cite this
Robotic Knee Parameter Tuning Using Approximate Policy Iteration. / Gao, Xiang; Wen, Yue; Li, Minhan; Si, Jennie; Huang, He (Helen).
Cognitive Systems and Signal Processing - 4th International Conference, ICCSIP 2018, Revised Selected Papers. ed. / Fuchun Sun; Huaping Liu; Dewen Hu. Springer Verlag, 2019. p. 554-563 (Communications in Computer and Information Science; Vol. 1005).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Robotic Knee Parameter Tuning Using Approximate Policy Iteration
AU - Gao, Xiang
AU - Wen, Yue
AU - Li, Minhan
AU - Si, Jennie
AU - Huang, He (Helen)
PY - 2019/1/1
Y1 - 2019/1/1
N2 - This paper presents an online model-free reinforcement learning based controller realized by approximate dynamic programming for a robotic knee as part of a human-machine system. Traditionally, prosthesis wearers’ gait performance is improved by manually tuning the impedance parameters. In this paper, we show that the parameter tuning problem can be formulated as an optimal control problem and thus solved by dynamic programming. Toward this goal, we constructed an quadratic instantaneous cost, which resulted in a value function that could be approximated by a neural network. The control policy is then solved by the least-squared method iteratively, a framework of which we refer to as approximate policy iteration. We performed extensive simulations based on prosthetic kinetics and human performance data extracted from real human subjects. Our results show that the proposed parameter tuning algorithm can be readily used for adaptive optimal tuning of prosthetic knee control parameters and the tuning process is time and sample efficient.
AB - This paper presents an online model-free reinforcement learning based controller realized by approximate dynamic programming for a robotic knee as part of a human-machine system. Traditionally, prosthesis wearers’ gait performance is improved by manually tuning the impedance parameters. In this paper, we show that the parameter tuning problem can be formulated as an optimal control problem and thus solved by dynamic programming. Toward this goal, we constructed an quadratic instantaneous cost, which resulted in a value function that could be approximated by a neural network. The control policy is then solved by the least-squared method iteratively, a framework of which we refer to as approximate policy iteration. We performed extensive simulations based on prosthetic kinetics and human performance data extracted from real human subjects. Our results show that the proposed parameter tuning algorithm can be readily used for adaptive optimal tuning of prosthetic knee control parameters and the tuning process is time and sample efficient.
KW - Approximate dynamic programming (ADP)
KW - Lower limb prosthesis
KW - Policy iteration
KW - Sample efficient learning
UR - http://www.scopus.com/inward/record.url?scp=85065763048&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85065763048&partnerID=8YFLogxK
U2 - 10.1007/978-981-13-7983-3_49
DO - 10.1007/978-981-13-7983-3_49
M3 - Conference contribution
AN - SCOPUS:85065763048
SN - 9789811379826
T3 - Communications in Computer and Information Science
SP - 554
EP - 563
BT - Cognitive Systems and Signal Processing - 4th International Conference, ICCSIP 2018, Revised Selected Papers
A2 - Sun, Fuchun
A2 - Liu, Huaping
A2 - Hu, Dewen
PB - Springer Verlag
ER -