Tactile Execution Monitoring of Robotic Manipulation Via Time-Series Based Predictive Encoding
Florian Voigt, Abdeldjallil Naceri, Sami Haddadin
AI summary
Problem
Real-time, skill-level tactile anomaly detection is largely overlooked despite being critical for safe deployment in human-centered and industrial environments where strict safety and quality requirements apply.
Approach
TPEM segments tactile time-series data into a task-aligned skill frame to build a predictive distribution of expected pose and wrench signals, using a Z-score method to flag deviations in real time.
Key results
- Zero false positives across key insertion, peg-in-hole, and screw tightening tasks
- Consistently outperformed SVM, HMM, and LSTM-VAE baselines in skill-level monitoring
- Reliably detected obstructions, wrong keys, and friction changes under realistic sensor noise
- Enabled manipulator-agnostic, real-time execution monitoring via skill-frame formulation
Why it matters
Provides a robust, real-time safety framework for tactile robots, enabling reliable deployment in industrial assembly and human-robot collaboration.
Abstract
Tactile manipulation is a prominent and growing field where most research focuses on developing generalized manipulation policies. However, tactile execution monitoring - the ability to reliably evaluate manipulation at the skill level - is often overlooked, despite being critical for unsupervised de- ployment in both human-centered environments and industry, where strict safety and quality requirements apply. We propose the Tactile Predictive Encoding Model (TPEM), a time-series tactile perception framework inspired by human predictive encoding that enables real-time anomaly detection from skill- level sensory data. TPEM extends predictive coding concepts from global task modeling to precise monitoring of contact-rich manipulation beyond the capabilities of visual sensing. We evaluate TPEM on three representative tasks: key in- sertion and turning, peg-in-hole insertion, and screw insertion and tightening using an industrial assembly model. Experiments on a tactile-enabled Franka Emika robot under realistic noise conditions show robust anomaly detection with zero false pos- itives. Comparison with baseline methods - including Support Vector Machines (SVM), Hidden Markov Models (HMM), and recurrent generative models such as LSTM-VAE — demon- strates that TPEM consistently outperforms state-of-the-art approaches in contact-rich skill-level execution monitoring.