Empowering deaf communication: a novel LSTM model for recognizing Indonesian sign language

Abstract

Sign language plays a pivotal role in facilitating communication for the deaf community, bridging the gap with the broader society. Nevertheless, mastering sign language poses significant challenges due to the intricate nuances of body movements, hand gestures, and facial expressions. Sign language recognition technology is a pivotal solution aimed at enabling clear communication between deaf individuals and the wider community, thereby reducing the risk of miscommunication. This study introduces an innovative approach to address these challenges. We focus on the recognition of Indonesian Sign Language using a skeleton-based method, harnessing the capabilities of MediaPipe to extract critical hand and pose key points from sign language videos. The core of our approach involves the implementation of a long short-term memory (LSTM) model, which has showcased exceptional promise in accurately interpreting BISINDO. The proposed LSTM architecture excels with a remarkable validation accuracy of 92.857%, surpassing the accuracy and computational efficiency of previously proposed LSTM models. This significant advancement in technology propels us closer to bridging the communication gap between the deaf community and the broader population.