MINT: A Vision-Based Soft Sensor for Mutual Integration of Normal Interaction Force and Texture Perception
Mohammad Rafiee Javazm, Siddhartha Kapuria, Ozdemir Can Kara, Sonika Kiehler, Rami Hamada, Joga Ivatury, Farshid Alambeigi
AI summary
Problem
Current tactile sensors either capture texture or measure force, but not both directly, and existing hybrid approaches require labor-intensive, sensor-specific AI training and calibration.
Approach
The authors embedded a liquid-metal spiral resistive channel between the gel and mirror layers of a standard vision-based tactile sensor, enabling direct electrical force readouts alongside visual texture capture, with physical occlusion removed via prompt-driven generative inpainting.
Key results
- Novel hybrid fabrication integrating a spiral resistive channel within a VTS gel layer
- Direct normal force measurement via simple electrical calibration
- Prompt-driven generative inpainting to remove sensor occlusion from texture images
- Experimental validation identifying optimal spiral geometry balancing texture quality and force range
Why it matters
Offers a scalable, low-calibration tactile sensing platform for robotics and haptics that eliminates the need for large-scale AI training while capturing both force and texture simultaneously.
Abstract
Inspired by the design of Vision-based Tactile Sensors (VTSs) and soft resistive strain sensors, in this paper, we introduce MINT: a vision-based soft sensor for Mutual Integration of Normal interaction force and Texture perception. MINT is a hybrid vision-based tactile sensor that simultane- ously integrates normal force measurement with high-resolution texture perception. This unique sensor utilizes a soft resistive strain sensor between the Gel Layer and Mirror Layer of a typical VTS. By combining electrical and visual sensing modalities, MINT overcomes the limitations of existing resistive sensors and VTSs, offering a robust, efficient, and scalable solution for direct measurement of force and texture capture. To evaluate MINT’s functionality, we first propose a unique design and fabrication procedure. Next, we conduct a series of experiments, evaluating its force and texture sensing capabilities through interactions with various rigid objects.