Reliability of Mobile Camera-Based Hand Sign Recognition in Outdoor Environments

Hand sign recognition systems have the potential to allow intuitive visual communication between humans and robots. Current recognition models often lack validation in outdoor settings, and thus fall short toward field deployment on mobile platforms. In this work, we assess the precision and recall of hand sign recognition models in the field considering robot movement, distance and viewing angle of the human from the mobile vision system. Supervised models were cus- tom trained on skeletal hand data and their F1 scores were compared with various available pre-trained models at varying distances and viewing angles. Statistical analysis presented in this paper shows that the distance to subject from the vision system had a statistically significant impact on hand sign recognition in outdoor environments with mobile robots, while the impact of viewing angle remained insignificant with the models tested.