Predicting Interaction Shape of Soft Continuum Robots Using Deep Visual Models

Soft continuum robots, characterized by their inherent compliance and dexterity, are increasingly pivotal in applications requiring delicate interactions with the envi- ronment such as the medical field. Despite their advantages, challenges persist in accurately modeling and controlling their shape during interactions with surrounding objects. This is because of the difficulty in modeling the large degrees of freedom in soft-bodied objects that become more active during interactions. In this study, we present a deep visual model to predict the interaction shapes of a soft continuum robot in contact with surrounding objects. By formulating this task as a forward-statics problem, the model uses the initial state images containing the object configuration and future actuation values to predict interactive state images of the robot under this actuation condition. We developed and tested the model in both simulated and physical environments, explored the model’s predictive capabilities using monocular and binocular views, and tested the model’s generalization ability on different datasets. Our results show that deep learning methods are a promising tool for solving the complex problem of predicting the shape of a soft continuum robot interacting with the environment, requiring no prior knowledge about the system dynamics and explicit mapping of the environment. This study paves the way for future explorations in robot-environment interaction modeling and the development of more adaptable interaction shape control strategies.