Abstract
A speech impairment limits a person’s capacity for oral and auditory communication. A great improvement in communication between the deaf and the general public would be represented by a real-time sign language detector. Recent studies have contributed to promising progress in motion and gesture identification processes using DL methods and computer vision. But the development of static and dynamic sign language recognition models is still a challenging area of research. The difficulty is in obtaining an appropriate model that addresses the challenges of continuous signs that are independent of the signer. Different signers’ speeds, durations, and many other factors make it challenging to create a model with high accuracy and continuity. This study mainly focused on sign language recognition using a modified deep learning (DL) and hybrid optimization approach. With the Visual Geometry Group 16 (VGG16), spatial and geometric-based features are extracted, and motion features are extracted via the optical flow approach. A new DL model, CNNSa-LSTM, is a combination of a convolutional neural network (CNN), self-attention (SA), and long-short-Term memory (LSTM) to identify sign language. A hybrid optimizer (HO) is proposed using the hippopotamus optimization algorithm (HOA) and the pathfinder algorithm (PFA). The proposed model has been implemented in Python, and it has been evaluated over the existing models in terms of accuracy, sensitivity, specificity, word error rate (WER), sign error rate (SER), and normalized discounted cumulative gain (NDCG) as well. The proposed model has recorded the highest accuracy of 98.7%.