[Turkiye Chapter] AI-Generated Closed Captioning in Movies for Deaf and Hard-of-Hearing (HoH) Individuals

Challenge Background

Omdena-Turkiye and Dem Association (Dem Dernegi) will start an AI challenge for Omdena Local Chapters Impact Program. Dem Association is an NGO that works for equal opportunities for deaf and hard-of-hearing (HoH) people in Turkiye. Currently, they run an advocacy campaign for deaf and HoH individuals' equal access to audiovisual products, especially films/movies, with closed captions. Closed captioning is a type of translation in which auditory elements that will make the video products meaningful are included in the subtitle so that deaf and hard-of-hearing individuals can effectively consume the product.

The Problem

Currently, films/movies in Turkiye usually do not have closed captions. For example, in September 2022, only 1 of the 746 films shown/released on television, cinema, digital platforms, festivals, and events had closed captions. Dem Association aims to add "closed captions" to these films/movies for individuals with hearing loss, whose number is 3 million 721 thousand 954 in Turkiye.  

In addition to the problem of lack of closed captions, the existing closed captions are usually added to movies manually. It means a translator watches and listens to movies and prepares the closed captions. Existing "speech-to-text" tools can not distinguish different and overlapping human voices and background sounds. Also, they can not put them in writing appropriately for deaf culture. Omdena Turkiye Local Chapter partners with Dem Association to work on solving this problem by developing an AI-generated automated closed captioning system.

Goal of the Project

1. To find data sources for closed captioning AI systems
2. To develop a toy model of an AI-generated closed captioning system in English
3. To develop a toy model of an AI-generated closed captioning system in Turkish.

Project Timeline

What you'll learn