Enhancing Prosthetic Safety and Environmental Adaptability: A Visual-Inertial Prosthesis Motion Estimation Approach on Uneven Terrains

Environment awareness is crucial for enhancing walking safety and stability of amputee wearing powered prosthesis when crossing uneven terrains such as stairs and obstacles. However, existing environmental perception systems for prosthesis only provide terrain types and corresponding parameters, which fail to prevent potential collisions when crossing uneven terrains and may lead to falls and other severe consequences. In this paper, a visual-inertial motion estimation approach is proposed for prosthesis to perceive its movement and the changes of spatial relationship between the prosthesis and uneven terrain when traversing them. To achieve this, we estimate the knee motion by utilizing a depth camera to perceive the environment and align feature points extracted from uneven terrains. Subsequently, an error-state Kalman filter is incorporated to fuse the inertial data into visual estimations to obtain a more robust and accurate estimation, which is then utilized to derive the motion of the whole prosthesis for our prosthetic control scheme. Experiments conducted on our collected dataset and stair walking trials with powered prosthesis show that the proposed method can accurately track the motion of human leg and the prosthesis with the average root-mean-square error of toe trajectory less than 5 cm. The proposed method is expected to enable the environmental adaptive control for prosthesis, thereby enhancing amputee’s safety and mobility in uneven terrains.