Motion Generation for Surgical Robots Using Task-Aware Attention Based on Deep Predictive Learning

In this study, we propose a method for au- tonomously operating a surgical robot by controlling visual attention using the robot’s own state in deep predictive learn- ing. The proposed method, named TAIRNN (Task-Attentive Informed Recurrent Neural Network) uses a state-conditioned Query to retrieve visual Keys to deep predictive model. Ex- perimental results of point-to-point movement showed that this method can reach the final target with improved accuracy and in fewer steps compared to conventional methods that rely solely on image information. The results demonstrate that an approach that incorporates a robot’s self-state awareness into its visual attention mechanism is effective in suppressing task- irrelevant visual noise and improving control stability.