Visualisable and Adjustable Command Spaces for Gesture-Based Home Appliance Operation System Via HoloLens2

We deal with a visualisable and adjustable gesture-based system for operating home appliances with ges- tures that can be easily operated and adjusted by anyone. The system uses “Command Spaces” in which commands for home appliance operation are tied to a space. These command spaces are constructed using body-relative coordinates using gestures at the start of the operation, which enables operation regardless of the posture and location of the operator. Dif- ferent manners people are comfortable performing gestures in different postures. Gestures vary from person-to-person. In this paper, we construct a system that visualizes them using Mixed Reality via the Microsoft HoloLens2. This allows the operator to freely visualize and adjust them, improving the usability of the appliance operation system. Comparative experiments showed that operation time, operation accuracy, and usability were improved when the system was used. This shows that the system facilitates learning about home appliance operation.