Adaptive Diffusion Constrained Sampling for Bimanual Robot Manipulation
Haolei Tong, Yuezhe Zhang, Sophie C. Lueth, Georgia Chalvatzaki
AI summary
Problem
Traditional constrained sampling methods struggle with high-dimensional, multi-modal distributions in bimanual manipulation due to reliance on local gradients and fixed constraint weights. This limits their ability to handle complex, interdependent spatial and kinematic constraints in unstructured environments.
Approach
The authors propose ADCS, a diffusion-based framework that learns constraint-aware energy functions and uses a Transformer to dynamically compose constraint weights at inference. A two-stage batch-wise sampling strategy combines Langevin dynamics with density-aware resampling to efficiently generate diverse, constraint-satisfying joint configurations.
Key results
- Dynamically weights constraints via a Transformer without manual tuning
- Enables post-hoc adaptation to robot-specific constraints via differentiable chain rule
- Achieves higher sample diversity and constraint satisfaction in simulated bimanual tasks
- Outperforms baselines in task success, sampling efficiency, and generalization
Why it matters
Provides a scalable, adaptive planning foundation for complex multi-robot and bimanual manipulation tasks in unstructured environments.
Abstract
Coordinated multi-arm manipulation requires sat- isfying multiple simultaneous geometric constraints across high- dimensional configuration spaces, which poses a significant challenge for traditional planning and control methods. In this work, we propose Adaptive Diffusion Constrained Sampling (ADCS), a generative framework that flexibly integrates both equality (e.g., relative and absolute pose constraints) and struc- tured inequality constraints (e.g., proximity to object surfaces) into an energy-based diffusion model. Equality constraints are modeled using dedicated energy networks trained on pose differences in the Lie algebra space, while inequality constraints are represented via Signed Distance Functions (SDFs) and encoded into learned constraint embeddings, allowing the model to reason about complex spatial regions. A key innovation of our method is a Transformer-based architecture that learns to weigh constraint-specific energy functions at inference time, enabling flexible and context-aware constraint integration. Moreover, we adopt a two-stage batch-wise sampling strategy that improves precision and sample diversity by combining Langevin dynamics with resampling and density-aware re- weighting. Experimental results on dual-arm manipulation tasks show that ADCS significantly improves sample diversity and generalization in settings demanding precise coordination and adaptive constraint handling. Our website is made publicly available at: thomasston.github.io/ADCS.github.io/