Reformulating AI-Based Multi-Object Relative State Estimation for Aleatoric Uncertainty-Based Outlier Rejection of Partial Measurements
Thomas Jantos, Giulio Delama, Stephan Weiss, Jan Steinbrener
AI summary
Problem
Fusing AI-based 6-DoF pose measurements in EKFs traditionally requires inverting the measurement, which couples rotation errors into position estimates and prevents selective outlier rejection. Fixed measurement covariance matrices also fail to capture the dynamic uncertainty inherent in deep neural network predictions.
Approach
The authors reformulate the EKF to use direct object-relative pose measurements, decoupling translation and rotation. They replace the fixed measurement covariance with the DNN's predicted aleatoric uncertainty to dynamically reject only the unreliable translation or rotation components of a measurement.
Key results
- Eliminates measurement inversion, decoupling position and rotation estimates in the EKF
- Reduces position RMSE under noisy rotation conditions compared to inverse formulations
- Enables dynamic rejection of partial translation or rotation measurements based on aleatoric uncertainty
- Improves state estimation consistency and robustness in ambiguous or symmetric object scenarios
Why it matters
Provides a practical framework for deploying reliable AI-based perception in real-time robot navigation by dynamically adapting to measurement confidence.
Abstract
Precise localization with respect to a set of objects of interest enables mobile robots to perform various tasks. With the rise of edge devices capable of deploying deep neural networks (DNNs) for real-time inference, it stands to reason to use artificial intelligence (AI) for the extraction of object- specific, semantic information from raw image data, such as the object class and the relative six degrees of freedom (6- DoF) pose. However, fusing such AI-based measurements in an Extended Kalman Filter (EKF) requires quantifying the DNNs’ uncertainty and outlier rejection capabilities. This paper presents the benefits of reformulating the mea- surement equation in AI-based, object-relative state estimation. By deriving an EKF using the direct object-relative pose measurement, we can decouple the position and rotation mea- surements, thus limiting the influence of erroneous rotation measurements and allowing partial measurement rejection. Furthermore, we investigate the performance and consistency improvements for state estimators provided by replacing the fixed measurement covariance matrix of the 6-DoF object- relative pose measurements with the predicted aleatoric un- certainty of the DNN.