MoiréTac: A Dual-Mode Visuotactile Sensor for Multidimensional Perception Using Moiré Pattern Amplification
KIT-WA SOU, Junhao Gong, Shoujie Li, Chuqiao Lyu, Ziwu Song, Shilong Mu, Wenbo Ding
AI summary
Problem
Existing visuotactile sensors rely on sparse marker arrays that limit spatial resolution and lack clear analytical force-to-image relationships, while often sacrificing optical clarity for tactile feedback.
Approach
The sensor overlays two micro-gratings to convert microscopic deformations into continuous moiré fields, combining physics-based pattern features with deep learning for interpretable 6-axis force/torque regression and transparent vision.
Key results
- R² > 0.98 across all six force/torque axes with minimal cross-talk
- Threefold sensitivity tuning via grating geometric parameters
- Clear color perception and object classification maintained through moiré overlay
- Successful robotic cap removal with coordinated force/torque control
Why it matters
Provides a compact, physics-informed dual-mode sensing solution that bridges high-fidelity tactile feedback with visual context, advancing dexterous robotic manipulation.
Abstract
Visuotactile sensors typically employ sparse marker arrays that limit spatial resolution and lack clear analytical force-to-image relationships. To solve this problem, we present Moir ́eTac, a dual-mode sensor that generates dense interference patterns via overlapping micro-gratings within a transparent architecture. When two gratings overlap with mis- alignment, they create moir ́e patterns that amplify microscopic deformations. The design preserves optical clarity for vision tasks while producing continuous moir ́e fields for tactile sensing, enabling simultaneous 6-axis force/torque measurement, contact localization, and visual perception. We combine physics-based features (brightness, phase gradient, orientation, and period) from moir ́e patterns with deep spatial features. These are mapped to 6-axis force/torque measurements, enabling inter- pretable regression through end-to-end learning. Experimental results demonstrate three capabilities: force/torque measure- ment with R2>0.98 across tested axes; sensitivity tuning through geometric parameters (threefold gain adjustment); and vision functionality for object classification despite moir ́e overlay. Finally, we integrate the sensor into a robotic arm for cap removal with coordinated force and torque control, validating its potential for dexterous manipulation.