Online 3D Edge Reconstruction of Wiry Structures from Monocular Image Sequences

Three-dimensional (3D) reconstruction of wiry structures from vision suffers from thin geometry, lack of texture, and severe self-occlusions. We propose an online 3D edge reconstruction framework that uses monocular image sequences to reconstruct the wiry structures whose skeletons are mainly straight as commonly found in the real world. To reconstruct such structures in an efficient manner, we employ straight edges constructed from points as underlying primitives of the representation. This is to address the harsh geometric nature of wiry objects (e.g., severe self-occlusion) and also to avoid a typically expensive line matching process. Specifically, we first construct sparse 3D points by tracking feature points, while simultaneously refining the camera poses via a robust maximum a posteriori (MAP) inference. These sparse points are then used to generate edge candidates and the belief of each candidate is updated in a Bayesian fashion using a likelihood evaluated on the image observation. Finally, we take the set of 3D edges with beliefs greater than a threshold and apply a post-processing step to reject false edges. We experimentally validate our framework using real-world wiry objects and demonstrate a manipulation task using the reconstruction. The proposed framework exhibits superior performance over state-of-the-art algorithms for the class of wiry structures and the potential to be easily used for subsequent robotic tasks.