Vision-Based Robotic Assembly from Novel Graphical Instructions

For the purpose of performing robotic assembly from a novel graphical instruction, this paper proposes a new method for aligning assembly parts based on the visual information guided by the image in a graphical instruction manual. Our proposed method comprises two phases: We first detect an assembly part drawn in the instruction manual and then estimate its relative pose among multiple assembly parts to be assembled. For the detection of assembly parts, we build the matching algorithm based on fast-directional chamfer matching (FDCM) by utilizing multiple images of actual assembly parts captured from multiple angles. For the relative pose estimation, we collect the poses of the identified parts and match them with the assembly scenes in the graphical instruction manuals. We conducted collaborative assembly experiments between a human and a robot for a set of furniture parts. We confirmed that the captured images matched the graphics contained in the assembly manual well. In addition, we confirm that, with the help of a human, a robot can efficiently assemble furniture with a novel instruction manual.