Semantic 3D Skeleton Extraction for Precision Agricultural Robotics : Preliminary Result
Dayeon Yang, Chanyoung Ju
AI summary
Problem
Recovering accurate branch topology from real-world orchard point clouds is challenging due to dense foliage, severe self-occlusion, and sensor noise, which cause conventional geometric methods to fail.
Approach
The authors fuse RGB-D and LiDAR data to create a multi-modal dataset and ground truth skeletons, then train a Conditional Flow Matching model to progressively map noisy point cloud nodes to accurate skeleton positions, enforced by a Minimum Spanning Tree.
Key results
- Multi-modal leaf-off orchard dataset with fused RGB-D and LiDAR point clouds
- ADTree-derived ground truth skeletons for supervised training
- Conditional Flow Matching framework for robust 3D skeleton node prediction
- MST-based topology enforcement ensuring valid, connected branch structures
Why it matters
Provides a robust foundation for automated pruning, growth monitoring, and harvesting systems in complex, real-world orchard environments.
Abstract
A multi-modal dataset was constructed in a real orchard environment under leaf-off conditions using an RGB-D camera and LiDAR, enabling clear observation of branch and trunk structures. The complementary geometric information from both sensors allows for more precise 3D structural reconstruction. Dense point clouds obtained from the RGB-D camera are fused with LiDAR point clouds via ICP registration, followed by ground removal and DBSCAN clustering to seg- ment individual trees. AdTree is then applied to each segmented tree to extract the 3D skeletal structure and generate Ground Truth. The constructed GT explicitly represents the hierarchical branch structure of each tree, and additional data collection under leaf-on conditions is planned to enable quantitative evaluation of skeleton extraction performance across varying foliage conditions. Furthermore, the constructed dataset will be utilized for training and evaluation of a Flow Matching- based generative model for tree skeletonization. Flow Matching enables stable skeleton reconstruction even from noisy and heavily occluded point clouds in real orchard environments, and the dataset is expected to facilitate quantitative analysis of performance differences between leaf-off and leaf-on conditions.