← Back ICRA 2026

Zero-Shot Sim2Real Transfer for Magnet-Based Tactile Sensor on Insertion Tasks

Beining Han, Abhishek Joshi, Jia Deng

PDF

AI summary

Key figure (auto-extracted from paper)

GCS enables zero-shot sim-to-real transfer of reinforcement learning policies for blind insertion tasks using raw, dense tactile readings, outperforming prior methods by nearly 50%.

sim-to-real transfer tactile sensing reinforcement learning magnet-based sensors blind insertion domain randomization

Problem

The large sim-to-real gap for magnet-based tactile sensors prevents robots from learning contact-rich manipulation skills in simulation and deploying them directly on real hardware, while existing binarization techniques discard the dense force information required for precise tasks like insertion.

Approach

GCS bridges the gap by simulating non-uniform contact with Gaussian bumps, approximating the sensor's Poisson effect via convolution, and applying domain randomization to force scaling factors during training.

Key results

Enables zero-shot sim-to-real transfer for blind peg-in-hole insertion tasks
Achieves 80% average real-world success rate, surpassing baselines by ~50%
Demonstrates effective use of raw, dense 3-axis tactile readings without binarization
Proves all three GCS components are necessary for successful real-world deployment

Why it matters

Allows robots to leverage affordable, dense tactile sensors for complex manipulation tasks without costly real-world data collection or information-losing preprocessing.

Abstract

Tactile sensing is an important sensing modality for robot manipulation. Among different types of tactile sensors, magnet-based sensors, like u-skin, balance well between tactile density, high-durability, and compactness. However, the large sim-to-real gap of tactile sensors prevents robots from acquiring useful tactile-based manipulation skills from simulation data, a recipe that has been successful for achieving complex and sophisticated control policies. Prior work has used binarization techniques to bridge the sim-to-real gap for dexterous in-hand manipulation with magnet-based sensors. However, binarization inherently loses much information that is useful in many other tasks, e.g., insertion. In our work, we propose GCS, a novel sim-to-real technique to learn contact-rich insertion skills with dense, distributed, 3-axes tactile readings from magnet-based tactile sensors. We evaluated our approach on blind insertion tasks and show successful zero-shot sim-to-real transfer of RL policies with raw tactile readings as input. Code is released at: https://github.com/princeton-vl/Tactile-GCS.

Index terms

Sensorimotor Learning Force and Tactile Sensing Contact Modeling