Introduction

Over 5% of the world’s population (466 million people) has disabling hearing loss. 4 million are children [1]. They can be hard of hearing or deaf. Hard of hearing people usually communicate through spoken language and can benefit from assistive devices like cochlear implants. Deaf people mostly have profound hearing loss, which implies very little or no hearing.

The main impacts of deaf is on the individual’s ability to communicate with others in addition to the emotional feelings of loneliness and isolation in society. Consequently, they can not equally access public services, mostly education and health and have not equal rights in participating at the active and democratic life. This leads to a negative impact in their lives and the lives of the people surrounding them. Over the world, deaf people often communicate using a sign language with gestures of both hands and facial expressions. Sign languages are full-fledged natural  languages with their own grammar and lexicon. However, they are not universal although they have striking similarities. In Morocco, deaf children receive very few education assistance. For many years, they were learning the local variety of sign language from Arabic, French, and American Sign Languages [2]. In April 2019, the governement standardized the Moroccan Sign Language (MSL) and initiated programs to support the education of deaf children [3]. However, the involved teachers are mostly hearing, have limited command of MSL and lack resources and tools to teach deaf learn from written or spoken text. Schools recruit interpreters to help the student understand what is being teached and said in class. Otherwise, teachers use graphics and captioned videos to learn the mappings to signs, but lack tools that translate written or spoken words and concepts into signs. This project comes to solve this particular issue.

Objectives

We propose an Arabic Sppech-to-MSL translator. The translation could be divided into two parts, the speech-to-text part and the text-to-MSL part. In a previous work [4], we was interested in the arabic Speech-to-Text translation. We conducted a research and a comparison on the existing Speech-to-Text APIs. A web application was built for this end [check web app here]. Our main focus in this current work is to take the results from the Speech-to-Text module and perform Text-to-MSL translation.

Up to now, there is not enough data of arabic words with their translations into MSL. This is of course challenging because we have to bring interpreters and linguists together in order to create this initial corpus. Each word or concept in the arabic corpus could be mapped to a time series of hand gestures and facial expressions in the MSL corpus. Our main objective is to find the best possible mapping between these two corpuses. The collected data will allow us to train big Deep Learning Models. In fact, we aim to explore the existing deep learning pretrained architectures suitable for analyzing arabic words and
sentences and find their mappings with the MSL encodings. Recurrent Neural Networks using GRU and LSTM units and their variants are proved to be the most suitable and powerful when dealing with sequential data [5][6]. We believe that tuning these state of the art models will allow us to achieve good generalization performances.

Expected outcomes

With this work we axpect building the MSL and the Arabic text corpuses that we aim keeping free and open for public use. For this end, we will develop an interactive web application for the creation of the two corpuses. The Text-to-MSL translation product will be hosted on a web and mobile application.