One-Shot Cross-Geometry Skill Transfer through Part Decomposition
Rory Thompson, Ondrej Biza, George Konidaris
AI summary
Problem
Existing skill transfer methods struggle to generalize to objects with unfamiliar or substantially different shapes, especially when only a single demonstration is available.
Approach
The method decomposes objects into semantic parts, uses generative shape models to transfer interaction points part-by-part, and optimizes a heuristic based on salient part relationships to adapt the skill to novel objects.
Key results
- Successfully transfers skills to a wider range of object geometries than whole-object warping and neural descriptor fields
- Achieves accurate one-shot transfer in both simulation and real-world pouring/placing tasks
- Introduces relational descriptors to resolve symmetry and local minima in part-based shape reconstruction
- Autonomously identifies and optimizes only the salient part relationships required for a given skill
Why it matters
Enables robots to quickly adapt to novel objects without extensive retraining, advancing practical one-shot imitation learning for real-world manipulation.
Abstract
Given a demonstration, a robot should be able to generalize a skill to any object it encounters—but existing approaches to skill transfer often fail to adapt to objects with unfamiliar shapes. Motivated by examples of improved transfer from compositional modeling, we propose a method for improving transfer by decomposing objects into their constituent semantic parts. We leverage data-efficient generative shape models to accurately transfer interaction points from the parts of a demonstration object to a novel object. We autonomously construct an objective to optimize the alignment of those points on skill-relevant object parts. Our method generalizes to a wider range of object geometries than existing work, and achieves successful one-shot transfer for a range of skills and objects from a single demonstration, in both simulated and real environments