Robustness-Aware Tool Selection and Manipulation Planning with Learned Energy-Informed Guidance
Yifei Dong, Yan Zhang, Sylvain Calinon, Florian T. Pokorny
AI summary
Problem
Robotic tool-use planning largely ignores robustness against external disturbances, prioritizing task completion over disturbance resilience. Selecting and using tools robustly is challenging due to varying tool geometries, complex contact dynamics, and the high computational cost of evaluating robustness.
Approach
The method uses a hierarchical optimization pipeline that first identifies the most robust tool-configuration pair and then plans a trajectory maintaining that robustness. A neural network trained offline on simulated escape-energy data provides fast, efficient robustness guidance during online planning.
Key results
- Jointly optimizes tool selection and contact-rich manipulation trajectories for disturbance resilience
- Introduces a learned energy-informed robustness metric derived from caging analysis for efficient online guidance
- Demonstrates consistent selection of robust tools and generation of disturbance-resilient plans across rigid, articulated, and deformable tasks
- Outperforms clearance-based and vision-language model baselines in simulation and real-world experiments
Why it matters
Enables robots to select and use tools reliably under uncertainty, advancing robust manipulation for real-world applications.
Abstract
Humans subconsciously choose robust ways of selecting and using tools, for example, choosing a ladle over a flat spatula to serve meatballs. However, robustness under external disturbances remains underexplored in robotic tool-use planning. This paper presents a robustness-aware method that jointly selects tools and plans contact-rich manipulation trajec- tories, explicitly optimizing for robustness against disturbances. At the core of our method is an energy-based robustness metric that guides the planner toward robust manipulation behaviors. We formulate a hierarchical optimization pipeline that first identifies a tool and configuration that optimizes robustness, and then plans a corresponding manipulation trajectory that maintains robustness throughout execution. We evaluate our method across three representative tool-use tasks. Simulation and real-world results demonstrate that our method consis- tently selects robust tools and generates disturbance-resilient manipulation plans.