Robotic Classification of Divers� Swimming States Using Visual Pose Keypoints As IMUs
Demetrious T. Kutzke, Ying-Kun Wu, Elizabeth Terveen, Junaed Sattar
AI summary
Problem
Wireless communication in water severely attenuates signals, preventing diver-worn health sensors from streaming data to autonomous underwater vehicles and hindering the detection of life-threatening medical emergencies.
Approach
The system uses a monocular camera to track 3D human joint keypoints over time, computes translational and rotational acceleration to create a pseudo-IMU feature stream, and applies time-series classifiers to detect transitions from swimming to motionless states.
Key results
- Novel pseudo-IMU feature extraction pipeline from monocular 3D pose keypoints
- Diverse underwater dataset capturing swimming-to-stationary state transitions
- Successful in-water classification of diver states using six time-series models
- Real-time state transition detection implemented onboard an AUV with LED feedback
Why it matters
This non-invasive, vision-based monitoring method bypasses underwater communication limits, providing a scalable solution for robotic dive buddies to detect medical emergencies and improve diver safety.
Abstract
Traditional human activity recognition uses either direct image analysis or data from wearable inertial mea- surement units (IMUs), but can be ineffective in challenging underwater environments. We introduce a novel hybrid ap- proach that bridges this gap to monitor scuba diver safety. Our method leverages computer vision to generate high-fidelity motion data, effectively creating a “pseudo-IMU” from a stream of 3D human joint keypoints. This technique circumvents the critical problem of wireless signal attenuation in water, which plagues conventional diver-worn sensors communicating with an autonomous underwater vehicle (AUV). We apply this system to the vital task of identifying anomalous scuba diver behavior that signals the onset of a medical emergency such as cardiac arrest—a leading cause of scuba diving fatalities. By integrating our classifier onboard an AUV and conducting experiments with simulated distress scenarios, we demonstrate the utility and effectiveness of our method for advancing robotic monitoring and diver safety.