Applying Multistep Classification Techniques With Pre-Classification to Recognize Static and Dynamic Hand Gestures Using a Soft Sensor-Embedded Glove

Abstract

Hand gestures have been widely used as an efficient information source for human-computer interaction (HCI) and apply to numerous fields, such as sign language translation and virtual reality (VR). Existing research on hand gesture recognition mainly considers either static gestures or dynamic gestures as gesture classes. However, simultaneously distinguishing both static and dynamic gestures is crucial for using hand gesture recognition systems in the real world because humans perform both types of gestures, not just one type in general. In this research, a multistep classification technique with a novel step called pre-classification is applied to simultaneously segment gesture patterns and classify static and dynamic gestures using a soft sensor-embedded glove, which measures the metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint angles of each finger every 15 ms. Specifically, the proposed hand gesture recognition system classifies gesture patterns as static or dynamic with pre-classification to handle both gestures with separately designed classifiers in one system for higher recognition performance. Combined with the gesture segmentation step, pre-classification shows reliable performance when a user performs various gesture types continuously. The proposed system’s effectiveness is validated through experiments with ten samples from eight subjects continuously performing eight static gestures and 11 dynamic gestures based on American Sign Language (ASL). The proposed system achieved 99.19% and 99.20% classification accuracy on static and dynamic gestures, respectively, and the lowest error in gesture segmentation than existing methods. This work can be applied to various application areas where high recognition performances are required, such as VR workforce training in manufacturing and medicine.