AI summary
Problem
Robots struggle to estimate the pose of movable objects in visually occluded environments because vision-based methods fail under occlusion and contact-rich exploration risks pushing objects out of the robot's workspace.
Approach
The method maintains a particle filter to track object pose uncertainty, computes real-time information gain using mutual information between potential trajectories and pose hypotheses, and integrates these into a closed-loop model predictive control framework that balances information gathering with reachability constraints.
Key results
- Novel particle filter belief framework for pose estimation using volumetric semantics
- Efficient real-time parallel computation of mutual information-based information gain
- Closed-loop MPC controller integrating information gain and reachability costs
- Consistent success in simulated and real-world rummaging tasks outperforming baselines
Why it matters
It advances autonomous manipulation in unstructured, cluttered environments where robots must actively interact with movable objects to overcome visual limitations.
Abstract
In this article, we present rummaging using mutual information (RUMI), a method for online generation of robot action sequences to gather information about the pose of a known movable object in visually occluded environments. Focusing on contact-rich rummaging, our approach leverages mutual infor- mation between the object pose distribution and robot trajectory for action planning. From an observed partial point cloud, RUMI deduces the compatible object pose distribution and approximates the mutual information of it with workspace occupancy in real time. Based on this, we develop an information gain cost function and a reachability cost function to keep the object within the robot’s reach. These are integrated into a model predictive control (MPC) framework with a stochastic dynamics model, updating the pose distribution in a closed loop. Key contributions include a new belief framework for object pose estimation, an efficient information gain computation strategy, and a robust MPC-based control scheme. RUMI demonstrates superior performance in both simulated and real tasks compared to baseline methods.