Environment-Driven Online LiDAR-Camera Extrinsic Calibration

LiDAR-camera extrinsic calibration (LCEC) is cru- cial for multi-modal data fusion in autonomous robotic systems. Existing methods, whether target-based or target-free, typically rely on customized calibration targets or fixed scene types, which limit their applicability in real-world scenarios. To address these challenges, we present EdO-LCEC, the first environment-driven online calibration approach. Unlike traditional target-free meth- ods, EdO-LCEC employs a generalizable scene discriminator to estimate the feature density of the application environment. Guided by this feature density, EdO-LCEC extracts LiDAR intensity and depth features from varying perspectives to achieve higher calibration accuracy. To overcome the challenges of cross-modal feature matching between LiDAR and camera, we introduce dual-path correspondence matching (DPCM), which leverages both structural and textural consistency for reliable 3D- 2D correspondences. Furthermore, we formulate the calibration process as a joint optimization problem that integrates global constraints across multiple views and scenes, thereby enhancing Received 14 July 2025; revised 17 September 2025; accepted 11 Octo- ber 2025. Date of publication 3 November 2025; date of current version 24 November 2025. This article was recommended for publication by Asso- ciate Editor Y. Sun and Editor P. Rocco upon evaluation of the reviewers’ comments. This work was supported in part by the National Natural Sci- ence Foundation of China under Grant 62473288, Grant 62233013, Grant 62272489, and Grant 62388101; in part by the National Key Research and Development Program of China under Grant 2025YFE0200003; in part by the Fundamental Research Funds for the Central Universities; in part by Xiaomi Young Talents Program; and in part by the National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University, under Grant HMHAI-202406. (Corresponding author: Rui Fan.) Zhiwei Huang, Jiaqi Li, Hongbo Zhao, and Wei Ye are with the Department of Control Science and Engineering, College of Electronic and Information Engineering, Tongji University, Shanghai 201804, China (e-mail: 2431985@tongji.edu.cn; 2251550@tongji.edu.cn; hongbozhao@ tongji.edu.cn; yew@tongji.edu.cn). Xiao Ma is with Beijing Institute of Aerospace Control Devices, Beijing 100039, China (e-mail: mx 169@126.com). Ping Zhong is with the School of Computer Science and Engineering, Central South University, Changsha, Hunan 410083, China, and also with the National Key Laboratory of Science and Technology on Automatic Target Recognition, National University of Defense Technology, Changsha, Hunan 410073, China (e-mail: ping.zhong@csu.edu.cn). Xiao-Hu Zhou is with the Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China (e-mail: xiaohu.zhou@ia.ac.cn). Rui Fan is with the College of Electronic and Information Engineering, Shanghai Institute of Intelligent Science and Technology, Shanghai Research Institute for Intelligent Autonomous Systems, Shanghai Key Laboratory of Intelligent Autonomous Systems, the State Key Laboratory of Autonomous Intelligent Unmanned Systems, and the Frontiers Science Center for Intelligent Autonomous Systems (Ministry of Education), Tongji University, Shanghai 201804, China, and also with the National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China (e-mail: rui.fan@ieee.org). This article has supplementary downloadable material available at https://doi.org/10.1109/TASE.2025.3627253, provided by the authors. Digital Object Identifier 10.1109/TASE.2025.3627253 overall accuracy. Extensive experiments on real-world datasets demonstrate that EdO-LCEC outperforms state-of-the-art meth- ods, particularly in scenarios involving sparse point clouds or partially overlapping sensor views. Note to Practitioners—This article presents an environment- driven approach for LiDAR-camera extrinsic calibration. Unlike conventional target-free methods, the proposed EdO-LCEC not only extracts matchable features from real-world scenes but also adapts its calibration strategy based on the environmental feature density. This environmental awareness significantly enhances cal- ibration robustness. By focusing on cross-modal feature matching and extrinsic optimization, our method performs reliably across various sensor configurations, including solid-state and mechani- cal LiDARs with differing fields of view and point densities. The proposed approach offers a practical and generalizable solution that improves upon existing target-free methods, facilitating deployment in sensor fusion and mechatronic systems. Our calibration software developed on EdO-LCEC will be publicly available at https://mias.group/EdO-LCEC.