Robotic Knee Prosthesis Real-Time Control Using Reinforcement Learning with Human in the Loop

Advanced robotic prostheses are expensive considering the cost of human resources and the time spent on manually tuning the high-dimensional control parameters for individual users. To alleviate clinicians’ effort and promote the advanced robotic prosthesis, we implemented an optimal adaptive control algorithm, which fundamentally is a type of reinforcement learning method, to automatically tune the high-dimensional control parameters of a robotic knee prosthesis through interaction with a human-prosthesis system. The ‘human-in-the-loop’ term means that the learning controller tunes the control parameters based on the performance of the robotic knee prosthesis while an amputee subject walking with it. We validated the human-in-the-loop auto-tuner with one transfemoral amputee subject for 4 hour-long lab testing sessions. Our results demonstrated that this novel reinforcement learning controller was able to learn through interaction with the human-prosthesis system and discover a set of suitable control parameter for the amputee user to generate near-normative knee kinematics.