Task and Skill Planning: Hierarchical Robot Planning with Black-Box Skills
Benned Hedegaard, Yichen Wei, Ziyi Yang, Ahmed Jaafar, Stefanie Tellex, George Konidaris, Naman Shah
AI summary
Problem
Traditional TAMP methods struggle to integrate heterogeneous, pre-existing robot skills like learned policies and force-controlled behaviors because they require explicit internal models or assume uniform controller structures.
Approach
The authors introduce TASP, which packages opaque skills as Composable Interaction Primitives (CIPs) that synthesize head and tail motion plans to safely transition between skills, allowing a hierarchical planner to compose them without needing to know their internal mechanics.
Key results
- Validated on real-world bimanual and mobile manipulator platforms
- Successfully plans long-horizon tasks mixing motion-planned, learned, and force-controlled skills
- Solves complex multi-room mobile manipulation problems with non-monotonic task structures
- Eliminates the need for explicit internal models of black-box skill controllers
Why it matters
Provides a practical pathway for deploying diverse, pre-existing robot skills in complex autonomous tasks without requiring custom controller modeling for each new capability.
Abstract
Task and motion planning (TAMP) is a well- established approach for solving long-horizon robot planning problems. Although TAMP methods have historically assumed that each task-level robot action, or skill, can be reduced to kinematic motion planning, recent work has explored integrat- ing closed-loop controllers and learned skills into TAMP-style systems. Our approach integrates pre-existing, heterogeneous robot skills—including learned, force-controlled, and black- box policies—into a hierarchical planner while preserving the object-centric failure reasoning of typical TAMP solvers. We leverage Composable Interaction Primitives (CIPs) to synthesize head and tail motion plans bridging consecutive skills, facilitat- ing both planning-time refinement and execution-time adjust- ment. We validate our Task and Skill Planning (TASP) approach through real-world experiments on a bimanual manipulator and a mobile manipulator, demonstrating that CIPs enable diverse robots to combine heterogeneous skills to solve complex, long-horizon tasks, including multi-room mobile manipulation problems with non-monotonic task structure.