MimuAI

Translator Screen
History screen
Settings

Inspiration

In a world bustling with conversations, not everyone has the ability to join in. Whether due to muteness or hearing impairments, millions are left on the sidelines of verbal communication, relying on sign language, a beautiful but often misunderstood language barrier. Witnessing the challenges of non-verbal communication in a predominantly verbal world, we were inspired to create a solution that leverages technology to make everyday interactions more inclusive and accessible.

What it does

MimuAI is an innovative app that captures live video and translates sign language into text and speech in real time. This transformative tool enables smooth and inclusive conversations, allowing individuals who do not understand sign language to communicate freely with those who use it.

How we built it

We have built a deep learning model using PyTorch and MMlab engines trained on Sign Language Detection data. This module is designed to capture video with sign language detection and convert it into text. Later on, we use MetaVoice to convert this text into speech. The data we submit is encoded with fully harmonic encryption. Finally, we combine all these components to develop a simple iOS application that uses the device camera as a source, allowing our deep learning model to process the captured video.

Challenges we ran into

The environment we are deploying for sign language detection is incompatible.
The delay in capturing text from the source is causing the application to run less smoothly.
Time management is proving to be a challenge.
We need to decide which features to develop and how to implement them.

Accomplishments that we're proud of

We are proud to have created an app that significantly improves communication accessibility for the deaf and mute communities. Our technology can change lives by fostering understanding and connection across language barriers.

What we learned

Throughout this project, we gained invaluable insights into the complexities of sign language and the importance of inclusivity in technology design. We also learned the power of teamwork and perseverance in overcoming technical challenges.

What's next for MimuAI

Looking ahead, we aim to expand our app's capabilities by including more sign languages and dialects, improving recognition accuracy, and developing features that cater to educational and professional settings. Our goal is to continue breaking down communication barriers and create a world where everyone can connect without limitations.