HEARAID

Inspiration

The idea for HearAid came from a simple question: Why should communication be difficult because of a hearing disability?, We realized that many deaf and hard-of-hearing people still struggle to communicate in classrooms, hospitals, workplaces, and even online meetings. We wanted to build a solution that could make these conversations more natural, accessible, and inclusive using AI.

What it does

HearAid is an AI-powered platform that listens to spoken language, converts it into text, translates it into sign language, and displays it through realistic 3D animations. It helps deaf and hard-of-hearing individuals understand conversations in real time, making communication easier in both physical and virtual environments.

How we built it

We built HearAid using React.js for the user interface, Unity WebGL and Blender for creating and rendering 3D sign language animations, and AI-powered speech recognition with NLP for translation. We also integrated Microsoft Graph API and Azure to support Microsoft Teams, enabling real-time accessibility during online meetings.

Challenges we ran into

Building HearAid wasn't easy. One of our biggest challenges was translating everyday sentences into proper sign language grammar instead of simply translating words one by one. We also spent a lot of time making the 3D animations look smooth and natural while ensuring the system responded quickly enough for real-time conversations.

Accomplishments that we're proud of

We're proud of building a working prototype that combines AI, speech recognition, NLP, and 3D animation into a single platform. Seeing spoken words transformed into sign language animations in real time was a rewarding milestone. We also successfully integrated our solution with Microsoft Teams to make online communication more inclusive.

What we learned

This project taught us much more than technical skills. We gained hands-on experience with React.js, Unity, Blender, NLP, Azure, and Microsoft Graph API while also learning the importance of designing technology that solves real-world accessibility problems. Most importantly, we understood how impactful inclusive technology can be.

What's next for HEARAID

We're excited to take HearAid even further. Our next steps include supporting multiple sign languages such as ISL and BSL, improving translation accuracy with more advanced AI models, adding offline support, enabling two-way communication through sign-to-speech translation, and exploring integration with smart glasses to make accessibility even more seamless.