Design and Evaluation of Real-Time Hand Gesture Interactions for Personal Indoor Wheeled Mobile Robots

Abstract

This paper designs a real-time hand gesture interaction system for controlling a personal indoor wheeled mobile robot and evaluates its usability. Previous research on controlling wheeled mobile robots through hand-gesture interaction has mainly focused on the technical aspects of hand gesture recognition. However, this study shifts the focus to the usability aspect. An elicitation study was conducted to identify common hand gestures used with wheeled mobile robots. Seven scenarios of using a personal indoor wheeled mobile robot were selected, and hand gestures were collected from 11 participants in their 20s and 30s. From these, one representative hand gesture for each scenario was chosen based on common features. For real-time interaction, hand shape detection was performed by fine-tuning a YOLO (You Only Look Once) model, and gesture recognition was achieved through post-processing using a pose estimation model. These recognized gestures were then mapped to control the movements of a wheeled mobile robot. To evaluate the real-time hand gesture recognition system, user evaluations were conducted in a controlled environment with a total of 40 participants. The NASA-TLX method was used to explore factors related to workload during gesture design, and the System Usability Scale (SUS) was used to evaluate the overall usability of the gesture control method. The workload evaluation revealed significant differences in the six NASA-TLX domains for different gestures. Additionally, the usability evaluation showed that the average SUS score was above the average, indicating that the gesture control method is considered usable. Interviews also revealed unexpected advantages, such as the emotional engagement users felt while using the gesture control method. Based on these analysis results, this study provides insights into the factors to consider when designing hand gestures for controlling personal indoor wheeled mobile robots and suggests that these findings could be applied to other types of land-based robots as well.