TY - GEN
T1 - A Data-Driven Reinforcement Learning Solution Framework for Optimal and Adaptive Personalization of a Hip Exoskeleton
AU - Tu, Xikai
AU - Li, Minhan
AU - Liu, Ming
AU - Si, Jennie
AU - Huang, He
N1 - Funding Information:
Research partly supported by National Science Foundation #1563454, #1563921, #1808752 and #1808898. We thank Sameer Amarnath Upadhye, Varun Nalam, Rajat Emanuel Singh, Abbas Alili and Chinmay Shah for their assistance in the experiments and Varun Nalam for editing the paper.
Funding Information:
*Research partly supported by National Science Foundation #1563454, #1563921, #1808752 and #1808898. (Corresponding author: He (Helen) Huang; email: hhuang11@ncsu.edu).
Publisher Copyright:
© 2021 IEEE
PY - 2021
Y1 - 2021
N2 - Robotic exoskeletons are exciting technologies for augmenting human mobility. However, designing such a device for seamless integration with the human user and to assist human movement still is a major challenge. This paper aims at developing a novel data-driven solution framework based on reinforcement learning (RL), without first modeling the human-robot dynamics, to provide optimal and adaptive personalized torque assistance for reducing human efforts during walking. Our automatic personalization solution framework includes the assistive torque profile with two control timing parameters (peak and offset timings), the least square policy iteration (LSPI) for learning the parameter tuning policy, and a cost function based on a transferred work ratio. The proposed controller was successfully validated on a healthy human subject to assist unilateral hip extension in walking. The results showed that the optimal and adaptive RL controller as a new approach was feasible for tuning assistive torque profile of the hip exoskeleton that coordinated with human actions and reduced activation level of hip extensor muscle in human.
AB - Robotic exoskeletons are exciting technologies for augmenting human mobility. However, designing such a device for seamless integration with the human user and to assist human movement still is a major challenge. This paper aims at developing a novel data-driven solution framework based on reinforcement learning (RL), without first modeling the human-robot dynamics, to provide optimal and adaptive personalized torque assistance for reducing human efforts during walking. Our automatic personalization solution framework includes the assistive torque profile with two control timing parameters (peak and offset timings), the least square policy iteration (LSPI) for learning the parameter tuning policy, and a cost function based on a transferred work ratio. The proposed controller was successfully validated on a healthy human subject to assist unilateral hip extension in walking. The results showed that the optimal and adaptive RL controller as a new approach was feasible for tuning assistive torque profile of the hip exoskeleton that coordinated with human actions and reduced activation level of hip extensor muscle in human.
KW - Data driven
KW - Exoskeleton
KW - Least square policy iteration
KW - Optimal adaptive control
KW - Reinforcement learning
UR - http://www.scopus.com/inward/record.url?scp=85111044782&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85111044782&partnerID=8YFLogxK
U2 - 10.1109/ICRA48506.2021.9562062
DO - 10.1109/ICRA48506.2021.9562062
M3 - Conference contribution
AN - SCOPUS:85111044782
T3 - Proceedings - IEEE International Conference on Robotics and Automation
SP - 10610
EP - 10616
BT - 2021 IEEE International Conference on Robotics and Automation, ICRA 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2021 IEEE International Conference on Robotics and Automation, ICRA 2021
Y2 - 30 May 2021 through 5 June 2021
ER -