Tactile-Conditioned Diffusion Policy for Force-Aware Robotic Manipulation
Erik Helmut, Niklas Wilhelm Funk, Tim Schneider, Cristiana de Farias, Jan Peters
AI summary
Problem
Most imitation learning approaches treat tactile feedback only as an observation, leaving applied forces as uncontrolled consequences of gripper commands rather than explicit targets.
Approach
The FARM framework uses GelSight Mini sensors and FEATS for force estimation, training a diffusion policy to jointly predict robot pose, grip width, and target grip force using a dual-mode control scheme.
Key results
- Outperformed baselines in plant insertion, grape picking, and screw tightening tasks
- Demonstrated success across high-force, low-force, and dynamic force adaptation scenarios
- Developed an open-source Actuated UMI gripper enabling direct transfer of human demonstrations without retargeting
- Achieved temporally consistent action sequences by jointly predicting pose, width, and force
Why it matters
Enables robots to safely handle fragile or deformable objects by explicitly regulating grasp forces to prevent slippage or breakage.
Abstract
Contact-rich manipulation depends on applying the correct grasp forces throughout the manipulation task, especially when handling fragile or deformable objects. Most existing imitation learning approaches often treat visuotactile feedback only as an additional observation, leaving applied forces as an uncontrolled consequence of gripper commands. In this work, we present Force-Aware Robotic Manipulation (FARM), an imitation learning framework that integrates high-dimensional tactile data to infer tactile-conditioned force signals, which in turn define a matching force-based action space. We collect human demonstrations using a modified version of the hand-held Universal Manipulation Interface (UMI) gripper that integrates a GelSight Mini visual tactile sensor. For deploying the learned policies, we developed an actuated variant of the UMI gripper with geometry matching our hand-held version. During policy rollouts, the proposed FARM diffusion policy jointly predicts robot pose, grip width, and grip force. FARM outperforms several baselines across three tasks with distinct force requirements—high-force, low- force, and dynamic force adaptation—demonstrating the ad- vantages of its two key components: leveraging force-grounded, high-dimensional tactile observations and a force-based control space. The codebase and design files are open-sourced and available at https://tactile-farm.github.io.