Sign Language Recognition and Production System for Indonesia

Challenge Background

The Deaf community in Indonesia is estimated to be quite large, with approximately 2.6 million people who are deaf or hard of hearing. These individuals are spread across the country, but challenges in accessibility, education, and employment are widespread. For example, only two-thirds of Indonesia's provinces have schools for the Deaf, and many children do not have access to education tailored to their needs​.

Additionally, while Indonesia has two recognized sign languages—Sistem Isyarat Bahasa Indonesia (SIBI) and Bahasa Isyarat Indonesia (BISINDO)—their use and support remain limited, which further complicates communication and accessibility for the Deaf community​.

The Problem

Current hearing accessibility tools, particularly in sign language recognition, primarily focus on alphabet-based recognition. Achieving conversational fluency requires more advanced word-level recognition, which is complex due to regional variations and detailed signing movements. There is a lack of AI solutions tailored to BISINDO, making it crucial to develop customised tools to support Indonesia’s Deaf community effectively.

Goal of the Project

1. Extract key points from sign language videos using computer vision tools like OpenPose or MediaPipe.
2. Apply data augmentation to expand sign language corpus for better accuracy.
3. Train a deep learning model (e.g., CNN-LSTM) to recognize hand gestures, and enable real-time sign language recognition.
4. Create algorithms for blending sign animations for natural flow, producing real-time visual output from text.

Project Timeline

What you'll learn

1. Experience in data collection, preprocessing, and augmentation for video-based datasets.
2. Mastery of deep learning techniques for sign language recognition. 
3. Development of animation systems for natural sign language production.
4. Skills in creating an integrated AI system for real-time output.