Low-Dimensional Tactile Glove for Visuo-Tactile Robot Hand Control: A Preliminary Study
Gi-gwang Baek, DongWook Kim
AI summary
Problem
Vision-only robot hand control fails to capture contact forces, slip, and occlusions, while existing high-cost tactile sensors hinder scalable dataset collection and deployment.
Approach
The authors designed a 20-sensor wearable glove mirroring a robot hand layout and paired it with a two-level learning framework that infers force distributions from sparse tactile signals and pretrains shared visuo-tactile representations via contrastive learning.
Key results
- Stable 300.03 Hz sampling with 20.1 µs jitter
- Noise-free baseline and uniform channel response
- Zero crosstalk across all 20 sensors
- Two-level framework for tactile-to-force inference and visuo-tactile pretraining
Why it matters
Provides a scalable, low-cost foundation for collecting human demonstration datasets and transferring tactile representations to robotic hands for dexterous manipulation.
Abstract
Dexterous control of multi-joint manipulators such as humanoid robot hands using vision alone faces fundamental limitations. Cameras cannot directly observe contact forces, slip, and deformation arising from robot-environment interactions, and are vulnerable to occlusion, making them insufficient for contact-rich manipulation tasks. Tactile sensing is therefore considered an essential component for dexterous manipula- tion. However, existing tactile-based approaches rely primarily on high-performance, high-cost sensors, imposing significant cost burdens in collecting human demonstration datasets and deploying tactile sensors on robot hands, which limits the scalability of tactile information in robotic systems. In this work, we present a low-dimensional wearable tactile glove as a scalable platform for visuo-tactile robot hand control, and propose a two-level learning framework built upon it. The tactile glove hardware consists of 20 FSR400 piezoresistive sensors, achieving stable 300 Hz data acquisition via dual multi- plexers, WiFi TCP communication, and clock synchronization. Hardware validation confirms 300.03 Hz sampling, 20.1 μs jitter, and noise-free signal quality, demonstrating that the system meets the requirements for tactile-based learning and control. The Level 1 framework investigates whether binary tactile signals alone can recover meaningful force distributions, aiming to show that low-cost, low-dimensional sensing can approxi- mate high-dimensional information through uncertainty-aware learning. In Level 2, 300 Hz tactile signals are combined with RGB camera input to pretrain a shared visuo-tactile repre- sentation via contrastive learning. Furthermore, by integrating the Level 1 framework into Level 2, we aim to explore whether high-quality shared visuo-tactile representations can be pretrained using low-dimensional tactile inputs through transformer-based models. Subsequent work will validate the pretrained representations through imitation learning and re- inforcement learning, and apply the full system to real-time physical robot hand control.