Fusing Satellite Imagery and Planimetric Maps for Cross-View Localization
Quang Long Ho Ngo, Zimin Xia, Alexandre Alahi
AI summary
Problem
Existing cross-view localization methods rely on single aerial modalities, typically satellite imagery, which degrade under occlusion and lack explicit semantic structure. Prior attempts to fuse complementary planimetric maps are either non-modular or fail to match state-of-the-art performance.
Approach
The authors introduce a plug-and-play fusion module that uses cross-modal deformable attention and a learned patch-level weighting rule to adaptively combine satellite and OpenStreetMap features per region and scale.
Key results
- Reduces mean localization error by 30.13% on the KITTI cross-area split to 3.85 m
- Consistently improves three state-of-the-art baseline models across VIGOR and KITTI benchmarks
- Demonstrates adaptive modality selection, leveraging planimetric maps under occlusion and satellite details elsewhere
- Mitigates cross-area overfitting and achieves state-of-the-art orientation recall
Why it matters
Enables more reliable and robust pose estimation for autonomous vehicles and robots by effectively leveraging widely available, complementary aerial data sources.
Abstract
Current cross-view localization methods predomi- nantly rely on satellite imagery as the aerial modality. Although recent work explores planimetric maps (e.g., OpenStreetMap tiles), these approaches often lag in performance. Yet both modalities are widely available and possess complementary properties. Satellite images are closer to ground-level camera imagery, offering finer detail, whereas planimetric maps contain annotated objects (e.g., streetlamps) and remain informative in areas where the ground is occluded, such as by foliage. Despite this, only one prior work provides an end-to-end method to fuse the two modalities, and it does not demonstrate their potential within state-of-the-art methods. To combine the strengths of both modalities, we propose a new fusion module that aug- ments standard encoders and demonstrates that integrating satellite imagery with planimetric maps improves state-of-the- art single-modality methods. The module comprises (i) cross- modal conditioning, which processes each modality’s encoding with awareness of the other, and (ii) a patch-level fusion rule that controls the granularity of information exchange. We achieve state-of-the-art results, reducing the mean localization error by 30.13%. Qualitatively, the fusion adaptively selects the more informative modality, improving overall accuracy. https://github.com/lipefree/cross-view-fusion