MOASIC: Skill-Centric Manipulation Planning with Physics Simulation
Itamar Mishani, Yorai Shaoul, Maxim Likhachev
AI summary
Problem
Planning long-horizon manipulation tasks is hindered by vast parameterized skill spaces and non-obvious intermediate steps, while existing methods rely on brittle symbolic abstractions or inefficient goal-directed search.
Approach
MOSAIC replaces goal-directed search with a multi-directional approach that uses a high-fidelity physics simulator to evaluate skill feasibility, constructing a graph of generator and connector skills guided by an oracle to discover physically grounded solutions.
Key results
- A feasibility-driven planning framework that anchors search in regions of high skill competence
- A physics-informed reasoning approach using in-the-loop simulation to ground planning decisions
- Extensive validation in simulation and real-world experiments demonstrating superior performance and scalability
- A domain-independent statistical oracle that balances exploration and exploitation for efficient skill composition
Why it matters
Enables general-purpose robots to flexibly compose imperfect, task-agnostic skills to solve complex manipulation tasks in unstructured environments.
Abstract
Planning long-horizon manipulation motions us- ing a set of predefined skills is a central challenge in robotics; solving it efficiently could enable general-purpose robots to tackle novel tasks by flexibly composing generic skills. Solutions to this problem lie in an infinitely vast space of parameterized skill sequences – a space where common incremental methods struggle to find sequences that have non-obvious intermediate steps. Some approaches reason over lower-dimensional, sym- bolic spaces, which are more tractable to explore but may be brittle and are laborious to construct. In this work, we introduce MOSAIC, a skill-centric, multi-directional planning approach that targets these challenges by reasoning about which skills to employ and where they are most likely to succeed, by utilizing physics simulation to estimate skill execution outcomes. Specifically, MOSAIC employs two complementary skill families: Generators, which identify “islands of compe- tence” where skills are demonstrably effective, and Connectors, which link these skill-trajectories by solving boundary value problems. By focusing planning efforts on regions of high competence, MOSAIC efficiently discovers physically-grounded solutions. We demonstrate its efficacy on complex long-horizon problems in both simulation and the real world, using a diverse set of skills including generative diffusion models, motion planning algorithms, and manipulation-specific models. Visit skill-mosaic.github.io for demonstrations.