Arabic Sign Language Recognition through Deep Neural Networks Fine-Tuning

Authors

  • Yaser Saleh Faculty of Information Technology, University of Petra, Amman, Jordan https://orcid.org/0000-0002-2257-721X
  • Ghassan Farid Issa Faculty of Information Technology, University of Petra, Amman, Jordan

DOI:

https://doi.org/10.3991/ijoe.v16i05.13087

Keywords:

Arabic Sign Language Recognition, Deep Learning, Fine tuning, Convolutional neural network

Abstract


Sign Language is considered the main communication tool for deaf or hearing impaired people.  It is a visual language that uses hands and other parts of the body to provide people who are in need to full access of communication with the world.  Accordingly, the automation of sign language recognition has become one of the important applications in the areas of Artificial Intelligence and Machine learning.  Specifically speaking, Arabic sign language recognition has been studied and applied using various intelligent and traditional approaches, but with few attempts to improve the process using deep learning networks.  This paper utilizes transfer learning and fine tuning deep convolutional neural networks (CNN) to improve the accuracy of recognizing 32 hand gestures from the Arabic sign language.  The proposed methodology works by creating models matching the VGG16 and the ResNet152 structures, then, the pre-trained model weights are loaded into the layers of each network, and finally, our own soft-max classification layer is added as the final layer after the last fully connected layer.  The networks were fed with normal 2D images of the different Arabic Sign Language data, and was able to provide accuracy of nearly 99%.

Author Biographies

Yaser Saleh, Faculty of Information Technology, University of Petra, Amman, Jordan

Yaser Saleh is an Assistant Professor at the University of Petra in Jordan. He holds a Ph.D. degree in Computer Science from Loughborough University. His research interests include Artificial Intelligence, Deep learning and Image processing.

Ghassan Farid Issa, Faculty of Information Technology, University of Petra, Amman, Jordan

Ghassan Issa is a professor of Computer Science at the University of Petra in Amman, Jordan. He received his BET from Toledo University, Ohio, BSEE from Trine University,Indiana, M.S and PhD. from Old Dominion University, Virginia. His research interests include the areas in Artificial Intelligence, Machine learning, and e-Learning System.

Downloads

Published

2020-05-14

How to Cite

Saleh, Y., & Issa, G. F. (2020). Arabic Sign Language Recognition through Deep Neural Networks Fine-Tuning. International Journal of Online and Biomedical Engineering (iJOE), 16(05), pp. 71–83. https://doi.org/10.3991/ijoe.v16i05.13087

Issue

Section

Papers