Estimating Deformable-Rigid Contact Interactions for a Deformable Tool Via Learning and Model-Based Optimization
Mark Van der Merwe, Miquel Oller, Dmitry Berenson, Nima Fazeli
AI summary
Problem
Classical rigid body contact models cannot adequately characterize interactions involving deformable tools, where tool deformation and object motion are complexly linked and partially observable.
Approach
A learned module predicts object motion and tool contact forces from partial point clouds, while a Contact Quadratic Program (CQP) resolves environment forces using physical priors like Coulomb friction and quasi-static equilibrium.
Key results
- Tool contact points estimated within 6mm on average
- Contact forces estimated within 0.2N on average
- Outperformed baselines across varying geometries, physical properties, and manipulation primitives
- Successful sim-to-real transfer demonstrated in real robot executions
Why it matters
Enables more performant and safe dexterous manipulation using soft robots or deformable tools by combining data-driven estimation with physical constraints.
Abstract
Dexterous manipulation requires careful reasoning over extrinsic contacts. The prevalence of deforming tools in human environments, the use of deformable sensors, and the increasing number of soft robots yields a need for approaches that enable dexterous manipulation through contact reasoning where not all contacts are well characterized by classical rigid body contact models. Here, we consider the case of a deforming tool dexterously manipulating a rigid object. We propose a hybrid learning and first-principles approach to the modeling of simultaneous motion and force transfer of tools and objects. The learned module is responsible for jointly estimating the rigid object’s motion and the deformable tool’s imparted contact forces. We then propose a Contact Quadratic Program to recover forces between the environment and object subject to quasi-static equilibrium and Coulomb friction. The results is a system capable of modeling both intrinsic and extrinsic motions, contacts, and forces during dexterous deformable manipulation. We train our method in simulation and show that our method outperforms baselines under varying block geometries and physical properties, during pushing and pivoting manipulations, and demonstrate transfer to real world interactions. Video results can be found at https://deform-rigid-contact.github.io/.