← Back ICRA 2026

M3TR: A Generalist Model for Real-World HD Map Completion

Fabian Immel, Richard Schwarzkopf, Frank Bieder, Jan-Hendrik Pauls, Christoph Stiller

PDF

AI summary

Key figure (auto-extracted from paper)

M3TR is a single generalist transformer model that completes outdated HD maps using sensor data and partial priors, matching specialized models while improving performance without priors.

HD Map Completion Generalist Model Map Prior Transformer Autonomous Driving Vectorized Perception

Problem

Offline HD maps quickly become outdated, and existing online map construction methods either ignore prior information, specialize in only one type of map change, or lack realistic evaluation benchmarks and metrics for partial map completion.

Approach

M3TR introduces a multi-masking map transformer with a novel query design and a generalist training regime that uses map masking as data augmentation, enabling it to handle any combination of outdated map priors and sensor data.

Key results

First comprehensive HD map completion benchmark with improved ground truth and prior-aware metric
Novel point query and query set design that fully utilizes prior map information
Generalist model matches specialized expert models across all prior scenarios without extra memory or compute
Achieves +1.4 mAP improvement without a prior and +4.3 mAP with priors on Argoverse 2 and nuScenes

Why it matters

Enables real-world deployment of autonomous vehicles by providing a single, robust model that dynamically adapts to partially outdated HD maps using live sensor data.

Abstract

Autonomous vehicles rely on HD maps for their operation, but offline HD maps eventually become outdated. For this reason, online HD map construction methods use live sensor data to infer map information instead. Research on real map changes shows that oftentimes entire parts of an HD map remain unchanged and can be used as a prior. We therefore introduce M3TR (Multi-Masking Map Transformer), a generalist approach for HD map completion both with and without offline HD map priors. As a necessary foundation, we address shortcomings in ground truth labels for Argoverse 2 and nuScenes and propose the first comprehensive benchmark for HD map completion. Unlike existing models that specialize in a single kind of map change, which is unrealistic for deployment, our Generalist model handles all kinds of changes, matching the effectiveness of Expert models. With our map masking as augmentation regime, we can even achieve a +1.4 mAP improvement without a prior. Finally, by fully utilizing prior HD map elements and optimizing query designs, M3TR outperforms existing methods by +4.3 mAP while being the first real-world deployable model for offline HD map priors. https://github.com/immel-f/m3tr

Index terms

Intelligent Transportation Systems Deep Learning for Visual Perception Computer Vision for Transportation