FlowSight: Vision-Based Artificial Lateral Line Sensor for Water Flow Perception
Tiandong Zhang, Rui Wang, Qiyuan Cao, Shaowei Cui, Gang Zheng, Shuo Wang
AI summary
Problem
Existing artificial lateral line sensors struggle to simultaneously measure local flow speed and direction while remaining compact, stable, and independent of bulky external equipment.
Approach
FlowSight mimics fish lateral line neuromasts with a flexible silicone tentacle that deforms in water flow; an internal camera captures these deformations, which a CNN-LSTM network processes to estimate flow velocity vectors.
Key results
- Novel vision-based artificial lateral line sensor design with integrated camera
- CNN-LSTM neural network for direct flow vector estimation from images
- Accurate flow speed and direction measurement validated in a controllable swim tunnel
- First closed-loop motion control of a bionic underwater robot using flow perception data
Why it matters
Provides underwater robots with a compact, reliable, and autonomous flow perception capability essential for navigation and operation in complex aquatic environments.
Abstract
This article presents a novel vision-based artificial lateral line (ALL) sensor, FlowSight, enhancing the perception capabilities of underwater robots. Through an autonomous vision system, FlowSight allows for simultaneous sensing the speed and direction of local water flow without relying on external auxiliary equipment. Inspired by the lateral line neuromast of fish, a flexible bionic tentacle is designed to sense water flow. Deformation and motion characteristics of the tentacle are modeled and analyzed using bidirectional fluid-structure interaction (FSI) simulation. Upon contact with water flow, the tentacle converts water flow information into elastic deformation information, which is cap- tured and processed into an image sequence by the autonomous vision system. Subsequently, a water flow perception method based on deep neural networks is proposed to estimate the flow speed and direction from the captured image sequence. The perception network is trained and tested using data collected from practical experiments conducted in a controllable swim tunnel. Finally, the FlowSight sensor is integrated into the bionic underwater robot RoboDact, and a closed-loop motion control experiment based on water flow perception is conducted. Experiments conducted in the swim tunnel and water pool demonstrate the feasibility and effectiveness of FlowSight sensor and the water flow perception method. Received 22 November 2024; revised 20 February 2025; accepted 21 April 2025. Date of publication 6 May 2025; date of current version 23 May 2025. This work was supported in part by the STI 2030—Major Projects under Grant 2022ZD0209600, in part by the National Natural Sci- ence Foundation of China under Grant 62403463, Grant 62276253, Grant 62203435, and Grant U23B2038, in part by the Postdoctoral Fellowship Program of CPSF under Grant GZC20241917, and in part by the China Postdoctoral Science Foundation under Grant 2024M763532. This article was recommended for publication by Associate Editor M. Kim and Editor S. Behnke upon evaluation of the reviewers’ comments. (Corresponding author: Rui Wang.) Tiandong Zhang, Rui Wang, and Shaowei Cui are with the State Key Lab- oratory of Multimodal Artificial Intelligence Systems, Institute of Automa- tion, Chinese Academy of Sciences, Beijing 100190, China (e-mail: tian- dong.zhang@ia.ac.cn; rwang5212@ia.ac.cn; shaowei.cui@ia.ac.cn). Qiyuan Cao is with the State Key Laboratory of Multimodal Artificial In- telligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China, and also with the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China (e-mail: caoqiyuan2021@ia.ac.cn). Gang Zheng is with the Centrale Lille, CRIStAL - Centre de Recherche en Informatique Signal et Automatique de Lille, University of Lille, 59000 Lille, France (e-mail: gang.zheng@inria.fr). Shuo Wang is with the State Key Laboratory of Multimodal Artificial Intelli- gence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China, also with the School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China, and also with the Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai 200031, China (e-mail: shuo.wang@ia.ac.an). This article has supplementary downloadable material available at https://doi.org/10.1109/TRO.2025.3567551, provided by the authors. Digital Object Identifier 10.1109/TRO.2025.3567551