Controlling FES of Arm Movements Using Physics-Informed Reinforcement Learning Via Co-Kriging Adjustment

Upper limb paralysis affects the quality of life. Functional Electrical Stimulation (FES) offers a solution to restore lost motor functions. Yet, there remain challenges in controlling FES to induce arbitrary arm movements. Rein- forcement learning (RL) emerges as a promising method for controlling arm movement with success in simulation. However, challenges remain in translating the successes into real-world settings. One dominant challenge is the sample efficiency of RL. This study presents a practical RL setup to control FES for arm movements. We also present a flexible method, called co-kriging adjustment (CKA), which combines a biomechanical simulator and real data to build an accurate model of the real system. We demonstrate our RL-based control on a 2-DoF planar setting where the subject’s arm, placed on a frictionless supporter, is stimulated to perform point-to-point reaching. By using 90 seconds of real interaction data, our RL-based control can perform the reaching with the average error over the workspace of 5.5 cm. Beyond the application of FES, our method can be extended to other control systems, propelling RL towards general uses in the real world.