View Synthesis and 6DoF Pose Estimation in mmWave Radar Neural Radiance Fields
Ahmad Kamari, Hemant Kumar, Nan Wu, Amirreza Hajrasouliha, Bo Han, Parth Pathak
AI summary
Problem
Vision-based pose estimation fails in poor lighting or adverse conditions, while existing mmWave radar solutions are limited to 2D localization, require LiDAR supervision, or cannot perform accurate 3D pose estimation from a single observation.
Approach
The system builds an implicit neural map from multi-view radar range-angle heatmaps and uses a particle filter to synthesize novel views, matching them against live radar data to compute real-time 6DoF pose.
Key results
- Achieves 0.713 SSIM for novel radar view synthesis
- Attains 0.34m translation and 17.15° rotation error for single-spectrum 6DoF estimation
- Reaches 0.66m and 0.81 rad Absolute Trajectory Error for continuous tracking
- Outperforms state-of-the-art radar SLAM without LiDAR supervision
Why it matters
Enables robust 3D localization and mapping for robotics and drones in visually degraded environments where optical sensors fail.
Abstract
Estimating a device’s 6DoF pose (i.e., location and orientation) within the environment is a fundamental problem in robotics, and beyond. Millimeter-wave (mmWave) radars have emerged as an attractive alternative to optical sensors (e.g., RGB cameras) in these tasks due to their ability to operate in poor lighting and adverse conditions such as smoke and fog. This paper presents mmNeRF, a view synthesis and 6DoF pose estimation system based on neural radiance fields (NeRF) designed specifically for mmWave radars. mmNeRF requires only radar range-angle heatmaps collected in a given environment to construct its implicit neural representation, ensuring multi-view consistency and producing high-quality view synthesis. It then builds a 6DoF pose estimation framework that queries the neural model with particle filters to perform scan & matching operations to yield an accurate 6DoF pose. We evaluate mmNeRF using over 50K radar frames collected in six different indoor environments on a handheld rig equipped with a radar. Our results show that mmNeRF achieves median translation and rotation errors of 0.34m and 17.15◦for single-spectrum 6DoF pose estimation and Absolute Trajectory Error (ATE) of 0.66m and 0.81 radians for continuous 6DoF pose tracking, considerably outperforming state-of-the-art solutions.