Stereo-Based Vision and Tactile Sensing for Robust Dual-Arm Robotic Connector Assembly
Michele Mirto, Alessio Caporali, Salvatore Pirozzi, Gianluca Palli
AI summary
Problem
Automating the insertion of flexible wire terminals into tight-tolerance connector housings is difficult due to object deformation and alignment challenges, which prior single-arm or fixed-fixture methods cannot reliably handle.
Approach
One arm refines the connector's pose using in-hand stereo vision, while the other manipulates the wire pin and uses tactile sensors to monitor contact forces and verify successful insertion in real time.
Key results
- Sub-millimeter refined 6D pose estimation of the connector via in-hand stereo vision
- Robust pin axial angle prediction with a confidence score that inversely correlates with error
- Real-time tactile insertion monitoring using a KNN classifier to verify mechanical locking
- Successful autonomous assembly of tight-tolerance connectors without fixed fixtures
Why it matters
Enables flexible, high-precision automation of wire harness manufacturing, reducing reliance on manual labor and rigid fixturing in industrial settings.
Abstract
The automation of Deformable Linear Object (DLO) manipulation remains a key challenge in industrial production. While prior works demonstrated reliable wire terminal insertion using vision and tactile sensing, they typically assume a fixed connector pose. This paper presents a dual- arm robotic system for fully autonomous connector assembly. A stereo vision system enables robust 6D pose estimation of the wire terminal, while a custom mechatronic gripper with inte- grated tactile sensing supports accurate insertion monitoring. In parallel, the second arm performs connector grasping. By combining complementary visual and tactile feedback across both manipulators, the system achieves the precision required for tight-tolerance insertion without fixed fixtures.