Deeper Introspective SLAM: How to Avoid Tracking Failures Over Longer Routes?

Large scale active exploration has recently revealed limitations of visual SLAM’s tracking ability. Active view planning methods based on reinforcement learning have been proposed to improve visual tracking robustness. In this work, we expose the limitations of deep reinforcement learning-based visual SLAM over longer routes. We demon- strate that additional modalities (depth, scene layout) offer little improvement. Furthermore, reward shaping is not the main reason behind the shortsightedness of the state-of-the-art visual SLAM tracker. We propose a novel video vision transformer- based architecture that improves the farsightedness of the visual tracker, which results in the completion of longer routes with efficient paths. Out of 60 challenging routes, our approach manages to complete 56 routes, which is a three-fold improvement over the state-of-the-art active view mapping (DI-SLAM) baseline. Interestingly, ORB-SLAM3 was unable to complete a single route without tracking failure. Our code is available at https: //tinyurl.com/w935spuz.