Vision Based Indian Sign Language Recognition Model
Keywords:
Hand Sign Recognition, Indian Sign Language, CNN, LSTM, Machine Learning, Vision Based Hand Sign Recognition, ISL, Long Short Term Memory (LSTM) , Convolutional Neural NetworkAbstract
Sign language (SL) is essential for deaf and hard-of-hearing people to communicate. However, these sign languages are not known to most healthy people. There is no universal language like verbally spoken languages as every country has its native language, so every country has its way of sign language. In India, we use Indian Sign Language (ISL). This survey provides an overview of the essential Indian sign language recognition and its translation work. Much research has been conducted in American Sign Language (ASL), but unfortunately, the same cannot be in the case of Indian Sign Language. There are different types, ways between sign language recognition processes worldwide. However, a few tasks are primarily similar, such as Pre-processing, feature extraction, and classification. The main focus of our proposed method is to design an ISL (Indian Sign Language) hand gesture motion translation tool for helping the deaf-mute community to convey their ideas by converting them to text format. We used a self-recorded ISL dataset for training the model for recognizing the gestures. CNN (Convolutional Neural Network) was used to extract the image features like skeletal features. LSTM (Long Short Term Memory) model was used to classify these gestures and then are translated into text.
Downloads
References
Saleh Aly and Walaa Aly. Deeparslr: A novel signer-independent deep learning framework for isolated arabic sign language gestures recognition. IEEE Access, 8:83199–83212, 2020.
Danilo Avola, Marco Bernardi, Luigi Cinque, Gian Luca Foresti, and Cristiano Massaroni. Exploiting recurrent neural networks and leap motion controller for the recognition of sign language and semaphoric hand gestures. IEEE Transactions on Multimedia, 21 (1):234–245, 2018.
Anusorn Chaikaew, Kritsana Somkuan, and Thidalak Yuyen. Thai sign language recognition: an application of deep neural network. In 2021 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunication Engineering, pages 128–131. IEEE, 2021.
Xin Chen, Jian Weng, Wei Lu, Jiaming Xu, and Jiasi Weng. Deep manifold learning combined with convolutional neural networks for action recognition. IEEE transactions on neural networks and learning systems, 29 (9):3938–3952, 2017.
Necati Cihan Camgoz, Simon Hadfield, Oscar Koller, and Richard Bowden. Subunets: End-to-end hand shape and continuous sign language recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 3056–3065, 2017.
Rong Ji. Research on basketball shooting action based on image feature extraction and machine learning.IEEE Access, 8:138743–138751, 2020.
Rashmi R Koli and Tanveer I Bagban. Human action recognition using deep neural networks. In 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), pages 376–380. IEEE, 2020.
Dimitrios Konstantinidis, Kosmas Dimitropoulos, and Petros Daras. A deep learning approach for analyzing video and skeletal features in sign language recognition. In 2018 IEEE International Conference on Imaging Systems and Techniques (IST), pages 1–6. IEEE, 2018.
Shan Li and Weihong Deng. Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial expression recognition. IEEE Transactions on Image Processing, 28 (1):356–370, 2018.
Yanqiu Liao, Pengwen Xiong, Weidong Min, Weiqiong Min, and Jiahao Lu. Dynamic sign language recognition based on video sequence with blstm-3d residual networks. IEEE Access, 7:38044–38054, 2019.
Guillaume Plouffe and Ana-Maria Cretu. Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE transactions on instrumentation and measurement, 65 (2):305–316, 2015.
LAI Qiuxia, Salman Khan, Yongwei Nie, Sun Hanqiu, Jianbing Shen, and Ling Shao. Understanding more about human and machine attention in deep neural networks. IEEE Transactions on Multimedia, 2020.
Chengcheng Wei, Jian Zhao, Wengang Zhou, and Houqiang Li. Semantic boundary detection with reinforcement learning for continuous sign language recognition. IEEE Transactions on Circuits and Systems for Video Technology, 31 (3):1138–1149, 2020.
Qinkun Xiao, Xin Chang, Xue Zhang, and Xing Liu. Multi-information spatial–temporal lstm fusion continuous sign language neural machine translation. IEEE Access, 8:216718–216728, 2020.
Chun-Yang Zhang, Yong-Yi Xiao, Jin-Cheng Lin, CL Philip Chen, Wenxi Liu, and Yu-Hong Tong. 3-d de-convolutional networks for the unsupervised representation learning of human motions. IEEE transactions on cybernetics, 2020.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Perspectives in Communication, Embedded-systems and Signal-processing - PiCES
This work is licensed under a Creative Commons Attribution 4.0 International License.
