CVF-DLO: Cross-Visual-Field Branched Deformable Linear Objects Route Estimation
Chang Yu, Jianjian Wang, Pingfa Feng, Dingwen Yu, Jianfu Zhang
AI summary
Problem
Perceiving and segmenting branched deformable linear objects (DLOs) is challenging due to complex crossings, bifurcations, and the difficulty of stitching multiple overlapping camera views into a coherent 3D route.
Approach
The framework extracts 2D DLO skeletons from images, pairs endpoints using a color/direction/curvature cost function, projects paths onto predefined physical surfaces using camera poses, and searches a restricted graph to reconstruct complete 3D routes across overlapping visual fields.
Key results
- Novel endpoint pairing cost function resolves complex crossings and bifurcations
- Cross-visual-field fusion accurately merges 3D path segments across viewpoints
- Graph-based route search reconstructs complete branched cable networks
- Validated on public and new BDLO datasets with real-time processing (>15 FPS)
Why it matters
Enables reliable automated inspection and manipulation of complex wiring harnesses and pipelines in robotics and industrial applications.
Abstract
The perception of deformable linear objects (DLOs) poses significant challenges in robotic manipulation. Crossovers, mergings, and bifurcations of multiple DLOs complicate the identification of individual DLO physical instances. Furthermore, DLOs are often too large to be captured by a single camera, requiring the stitching of multiple overlapping views. This paper presents CVF-DLO, a cross-visual-field route estimation framework of branched DLOs (BDLOs) laid along physical surfaces such as wire harnesses, based on images from multiple viewpoints and pose-aware cameras. CVF-DLO is applicable to various perception tasks involving DLO-like structures, such as verifying connection accuracy and route consistency in cables and pipes. We propose a DLO instance segmentation method that demonstrates superior performance in handling crossings and bifurcations. The extracted DLO paths are projected onto the designed cable-laying surfaces using the camera pose and scene model. Finally, DLO routes are retrieved by searching within the spatial path domain formed by intersecting visual fields. To validate our method on wiring harnesses and in- tersections, we use two public DLO datasets and introduce a new BDLO dataset to benchmark against state-of-the-art DLO instance segmentation methods. Additionally, we present a cabin wiring harness dataset to evaluate the performance of the cross- visual-field route estimation. We have released all our source code, supplementary video, and datasets (with ground truth) at https://github.com/ForNe-tech/CVF-DLO.