Hand gesture recognition with out-of-distribution gesture detection using a soft sensor embedded glove

Abstract

Hand gesture recognition has been applied to many applications, such as sign language translation and virtual reality. Soft sensor embedded gloves have been widely used to collect gesture data for hand gesture recognition. Using a soft sensor embedded glove has an advantage compared to vision-based approaches, because it is less affected by the environment and is not restricted to the angle of camera sensors, especially when it is applied to virtual reality industry. One of the existing challenges in machine learning-based hand gesture recognition is that new gestures, which are not seen in the training stage, are often discovered in the testing stage. In order to overcome this challenge, in this work, a hand gesture recognition model is proposed based on one-dimensional convolutional neural networks (1D CNN) and long short-term memory (LSTM) as well as a clustering method. In particular, a clustering method is used to detect out-of-distribution gestures, which are not contained in the clusters that are consisted of hand gestures used in the training stage. The experiment results validate that the proposed hand gesture recognition model performs better than existing hand gesture recognition methods and a proposed clustering methods detects out-of-distribution gestures with high accuracy.