Flow with the Force Field: Learning 3D Compliant Flow Matching Policies from Force and Demonstration-Guided Simulation Data
Tianyu Li, Yihan Li, Zizhe Zhang, Nadia Figueroa
AI summary
Problem
Contact-rich manipulation requires compliance, but existing policies either ignore force or require excessive human demonstrations and complex reward engineering.
Approach
The authors generate diverse force-informed simulation data from one demo using Laplacian editing and train a flow-matching policy that outputs both poses and impedance parameters for execution via a passive impedance controller.
Key results
- Lightweight force-informed data generation via virtual targets and Laplacian editing
- Adaptive Compliant Flow Matching policy integrating point clouds and force inputs
- Zero-shot transfer to real Franka robots for box flipping and bimanual grasping without real-world demos
- State-velocity field rollout scheme that reduces energy injection and improves safety
Why it matters
It drastically reduces the data burden for learning compliant robotic behaviors in complex, contact-heavy environments.
Abstract
While visuomotor policy has made advancements in recent years, contact-rich tasks still remain a challenge. Robotic manipulation tasks that require continuous contact demand explicit handling of compliance and force. However, most visuomotor policies ignore compliance, overlooking the importance of physical interaction with the real world, often leading to excessive contact forces or fragile behavior under uncertainty. Introducing force information into vision-based imitation learning could help improve awareness of contacts, but could also require a lot of data to perform well. One remedy for data scarcity is to generate data in simulation, yet computationally taxing processes are required to generate data good enough not to suffer from the Sim2Real gap. In this work, we introduce a framework for generating force-informed data in simulation, instantiated by a single human demonstration, and show how coupling with a compliant policy improves the performance of a visuomotor policy learned from synthetic data. We validate our approach on real-robot tasks, including non-prehensile block flipping and a bi-manual object moving, where the learned policy exhibits reliable contact maintenance and adaptation to novel conditions. Project Website: flow-with- the-force-field.github.io.