Virtual-Force Based Visual Servo for Multiple Peg-In-Hole Assembly with Tightly Coupled Multi-Manipulator
Jiawei Zhang, Chengchao Bai, Wei Pan, Jifeng Guo, Tianhang Liu
AI summary
Problem
Aligning multiple distant pegs and holes simultaneously for large-size parts is difficult for single manipulators, and existing methods lack effective visual feature integration for tightly coupled multi-manipulator systems during the hole-search stage.
Approach
The method uses monocular in-hand cameras to train neural networks for peg/hole state classification and positioning, then fuses multi-manipulator visual data through a virtual force model to guide cooperative motion control.
Key results
- 100% success rate in dual-manipulator dual peg-in-hole tasks with 0.2 mm clearance
- Robust performance against camera calibration errors
- Improved classification accuracy and positioning precision via varied hole appearance training
- Effective integration of multi-manipulator visual features using virtual forces
Why it matters
Provides a reliable vision-guided assembly solution for large-scale robotic tasks like space truss construction, advancing dual-arm automation in precision manufacturing.
Abstract
Multiple Peg-in-Hole (MPiH) assembly is one of the fundamental tasks in robotic assembly. In the MPiH tasks for large-size parts, it is challenging for a single manipulator to si- multaneously align multiple distant pegs and holes, necessitating tightly coupled multi-manipulator systems. For such MPiH tasks using tightly coupled multiple manipulators, we propose a collabo- rative visual servo control framework that uses only the monocular in-hand cameras of each manipulator to reduce positioning errors. Initially, we train a state classification neural network and a po- sitioning neural network. The former divides the states of the peg and hole in the image into three categories: obscured, separated, and overlapped, while the latter determines the position of the peg and hole in the image. Based on these findings, we propose a method to integrate the visual features of multiple manipulators using virtual forces, which can naturally combine with the coop- erative controller of the multi-manipulator system. To generalize our approach to holes of different appearances, we varied the appearance of the holes during the dataset generation process. The results confirm that by considering the appearance of the holes, classification accuracy and positioning precision can be improved. Finally, the results show that our method achieves 100% success rate in dual-manipulator dual peg-in-hole tasks with a clearance of 0.2 mm, while robust to camera calibration errors.