Coupled Particle Filters for Robust Affordance Estimation
Patrick Lowin, Vito Mengers, Oliver Brock
AI summary
Problem
Monolithic affordance estimators struggle with visual, geometric, and semantic ambiguities, leading to poor generalization and failure in real-world or challenging conditions like low light and clutter.
Approach
The method employs two coupled recursive particle filters that independently estimate graspable and movable regions, then cross-modally fuse their outputs to align attention on regions where both properties co-occur.
Key results
- Outperforms Where2Act, Hands-as-Probes, and HRP by 308%, 245%, and 257% in precision
- Maintains robust detection in dark and cluttered environments
- Achieves 70% success rate in real-world manipulation trials
- Cross-modal coupling resolves ambiguities better than individual measurement sources
Why it matters
Provides a reliable, embodiment-aware perception framework that enables robots to interact safely and effectively in unstructured real-world environments.
Abstract
Robotic affordance estimation is challenging due to visual, geometric, and semantic ambiguities in sensory input. We propose a method that disambiguates these signals using two coupled recursive estimators for sub-aspects of affordances: graspable and movable regions. Each estimator encodes property-specific regularities to reduce uncertainty, while their coupling enables bidirectional information exchange that focuses attention on regions where both agree, i.e., af- fordances. Evaluated on a real-world dataset, our method outperforms three recent affordance estimators (Where2Act, Hands-as-Probes, and HRP) by 308%, 245%, and 257% in pre- cision, and remains robust under challenging conditions such as low light or cluttered environments. Furthermore, our method achieves a 70% success rate in our real-world evaluation. These results demonstrate that coupling complementary estimators yields precise, robust, and embodiment-appropriate affordance predictions.