UnifyTalk

Inspiration

Communication should never be limited by someone's ability to speak, hear, see, or type. We were inspired by the everyday communication barriers faced by deaf, mute, and blind individuals and wanted to build a single platform that adapts to different ways of communicating.

What We Built

UnifyTalk is an AI-powered accessibility platform that brings multiple communication methods together in one place. It supports speech-to-text, text-to-speech, pictogram-based communication, quick phrases and SOS alerts, real-time sign language detection, voice navigation, and Braille output.

Instead of forcing everyone to communicate in the same way, UnifyTalk adapts communication to the user's needs.

How We Built It

The frontend was built using React 18, with Node.js and Express powering the backend. MongoDB Atlas handles data, Socket.io enables real-time communication, MediaPipe Hands powers real-time sign language detection, and the Web Speech API enables speech-based interaction. JWT is used for authentication and Cloudinary handles media.

The platform is structured into multiple accessibility-focused modules: Chat, Pictograms, Quick Phrases, Sign AI, Screen Reader, and Community.

Challenges We Faced

One of our biggest challenges was designing a system that could support very different accessibility requirements without making the interface complicated. We also had to integrate real-time sign detection, speech technologies, visual communication, and voice navigation into a unified experience.

Another challenge was making the experience intuitive enough that users can communicate quickly without having to understand the underlying technology.

What We Learned

Building UnifyTalk taught us that accessibility is not just about adding features—it is about designing technology around the user's needs. We learned how to combine AI-assisted interaction, real-time communication, accessibility-first UI design, and multiple input/output modalities into one platform.

What's Next

We plan to improve sign language recognition accuracy, expand supported sign languages, enhance offline accessibility, improve Braille and screen-reader support, and continue making UnifyTalk a more comprehensive communication platform.