Enhanced BERT-based Multi-Head Self-Attention Transformer for Transformation of Marathi Text to Marathi Sign Language Gloss-Reference-Cited by-同舟云学术

Enhanced BERT-based Multi-Head Self-Attention Transformer for Transformation of Marathi Text to Marathi Sign Language Gloss

Published:2024-08-08 Issue: Volume: Page:
ISSN:2375-4699
Container-title:ACM Transactions on Asian and Low-Resource Language Information Processing
language:en
Short-container-title:ACM Trans. Asian Low-Resour. Lang. Inf. Process.

Author:

waghmare Prachi¹^ORCID,deshpande Ashwini¹^ORCID

Affiliation:

1. Electronics and Telecommunication, Cummins College of Engineering for Women, Pune, India

Abstract

One recent advancement in the field of machine learning is the translation of text into sign language gloss which is a form of natural language for the deaf community. The work presented is a new method to translate Marathi text to Marathi sign language gloss by combining salient features of Bidirectional Encoder Representation from Transformer (BERT) for tokenization and complementing the tokenized frame with Encoder with Attention mechanism and decoding with the LSTM decoder. The work conducted experiments on the created corpora of Marathi text and Marathi sign language gloss sentence pairs. The experiments that employed 3 models show that the suggested model performs better than the existing approaches. The results show that the translation of Marathi text to sign gloss achieves improved performances with an accuracy of 91.5% and the BLEU scores BLEU-1: 85, BLEU-2: 75, BLEU-3: 65, and BLEU-4: 57.

Publisher

Association for Computing Machinery (ACM)

Link

https://dl.acm.org/doi/pdf/10.1145/3687304

Reference23 articles.

1. A Spanish speech to sign language translation system for assisting deaf-mute people

2. Marshall, Ian, and Éva Sáfár. 2003. A prototype text to British Sign Language (BSL) translation system. In The Companion Volume to the Proceedings of 41st Annual Meeting of the Association for Computational Linguistics, pp. 113-116.

3. Morrissey, Sara. 2008. Data-driven machine translation for sign languages. PhD diss., Dublin City University.

4. Othman Achraf and Mohamed Jemni. 2011. Statistical sign language machine translation: from English written text to American sign language gloss. arXiv preprint arXiv:1112.0168.

5. Khan, Nadeem Jadoon, Waqas Anwar, and Nadir Durrani. 2017. Machine translation approaches and survey for Indian languages. arXiv preprint arXiv:1701.04290Orbay, Alptekin, and Lale Akarun. 2020. Neural sign language translation by learning tokenization. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pp. 222-228. IEEE.