TacTip-based Dynamic Contact Force Estimation with Sequential Tactile Images and Its Applications to Robotic Force Tracking
Wantong Xie, Zhenyu Lu, Jingyang Liu, Jialong Yang, Lu Chen, Chenguang Yang
AI summary
Problem
Traditional force sensors are costly and prone to wear, while existing vision-based tactile methods rely on static calibration and degrade during dynamic interactions like slip.
Approach
The method processes sequential tactile images with a dynamic flow encoder to capture spatiotemporal features, then refines predictions using an exponentially weighted residual correction strategy before integrating them into an impedance control loop.
Key results
- Reduces mean absolute percentage error to 12.54% on dynamic slip data
- Decouples spatial deformations and temporal flow features via a dual-stream neural architecture
- Enables precise real-world robotic force tracking during high-speed slip interactions
- Maintains low inference latency (~4.17 ms) while outperforming baseline models
Why it matters
Provides a robust, low-cost alternative to physical force sensors for robots operating in dynamic, unstructured environments.
Abstract
Force estimation is crucial for robotics, human– machine interaction, and industrial automation. However, tra- ditional methods are often hindered by high cost, mechanical wear, and limited accuracy in dynamic scenarios. Vision-based tactile sensing provides a promising alternative, yet existing approaches commonly rely on static calibration and degrade under dynamic interactions such as slip. To overcome these limitations, we present a novel force prediction framework for TacTip sensors, termed as Frame-stack Force Prediction Method (FFPM). The framework integrates a Dynamic Tactile Flow Encoder to capture spatiotemporal features, enabling ac- curate modeling of dynamic force variations. An Exponentially Weighted Residual Correction strategy is further introduced to refine predictions by leveraging historical residuals, yielding smoother and more reliable force estimation. The predicted forces are incorporated into a force-tracking impedance control scheme, achieving precise tracking during slip interactions. Experiments on our constructed dataset demonstrate state-of- the-art performance, reducing MAPE to 12.54%, and further validate the effectiveness of the proposed framework in real- world dynamic force estimation and control.