Lightweight Hand-Waving Action Recognition Using Reservoir Computing in a Cafeteria Environment

Owing to the global labor shortage and increasing need for operational efficiency, the adoption of service robots is advancing rapidly. These robots must recognize human action to understand human intention and respond appropriately based on that understanding. The action recognition systems embedded in robots need to be lightweight to operate efficiently with limited computational resources. Reservoir computing (RC) is one of the solutions for lightweight action recognition systems. Yamaguchi et al. proposed an RC-based hand-waving recognition system; however, the system cannot process multiple persons simultaneously and works only when one person is in the image. Therefore, this study proposes a lightweight hand-waving recognition system that integrates OpenPose, StrongSORT, and RC to work in complex environments with multiple individuals. Experimental results demonstrated the effectiveness of the proposed system in processing multiple people simultaneously in a crowded environment and accurately recognizing hand-waving actions with 90.75% accuracy. We also confirmed that the proposed system can process data at 24-26 FPS. We demonstrated that the proposed system can perform real-time processing. In addition, the robot with the proposed system recognized hand-waving actions in the “Restaurant” task of RoboCup@Home 2024 and obtained the second-place score.