Enhancing Robot Perception Using Vision-Aware Cognition Model

In the field of robotics, the construction of ad- vanced perception models is essential for the successful exe- cution of complex tasks. Traditional perception models, often grounded in cognitive frameworks, fall short in adequately processing and interpreting visual data. There is a pressing need to enhance these models with advanced visual processing capabilities. The integration of sophisticated vision models with cognitive frameworks is expected to significantly enhance the performance of perception models, yet such integrations remain underexplored. In this paper, we propose a novel Vision-Aware Cognition Model that effectively merges visual and cognitive components to advance robot perception. Our model integrates a cognition model, which incorporates contextual memory for nuanced long-term memory and context comprehension, with a vision model that employs spatial attention to focus on key regions of visual input. This harmonious integration enables not only robust feature extraction but also heightened adaptability to visual environmental changes. We evaluated our model using a simulated robotic hand on a valve-turning manipulation task. By leveraging saliency visual- ization, we made the robot’s decision-making process transpar- ent, showcasing the distinct functions of the visual and cognitive components. The vision model demonstrates superior object segmentation, while the cognition model is adept at operation points tracking. By leveraging the strengths of both components, the proposed model achieves efficient hybrid feature extraction. Furthermore, we conducted quantitative evaluations of the model’s adaptability to various visual changes, which revealed statistically significant performance improvements, highlighting its remarkable capacity for enhancing robot perception.