UnifyConnect

UnifyConnect

Inspiration

We were inspired by modern computer vision technology and speech-to-text capabilities. With advancements in AI, we saw an opportunity to leverage these technologies to help disabled individuals communicate seamlessly. Our goal was to eliminate communication barriers and make conversations more accessible for everyone.

What It Does

Our web app enables real-time video calling with:

Live Captions – Converts speech into text for instant subtitles.
Sign Language Recognition – Detects and translates sign language into text.
Seamless Communication – Allows users to communicate naturally, whether they speak or sign.

How We Built It

We combined various technologies to bring this idea to life:

A custom-trained CNN-LSTM model to translate sign language into text with high accuracy.
MediaPipe for real-time hand movement tracking.
Speech-to-Text APIs to generate real-time captions.
React & Node.js to build the video calling platform.
WebRTC for peer-to-peer video communication.

Challenges We Ran Into

Training an accurate yet lightweight CNN-LSTM model for sign language recognition while ensuring real-time performance.
Optimizing hand tracking using MediaPipe to work efficiently across different lighting conditions.
Ensuring low-latency processing so users experience minimal delays in captions and sign translations.
Integrating multiple AI models smoothly without affecting video call performance.

Accomplishments That We're Proud Of

Successfully implementing real-time sign language recognition using our custom-trained CNN-LSTM model.
Achieving low-latency speech-to-text conversion for seamless conversations.
Building an inclusive platform that empowers disabled individuals to communicate more effectively.

What We Learned

How sign language works and the challenges faced by individuals who rely on it.
Optimizing CNN-LSTM models for real-time performance in a web environment.
Improving MediaPipe-based hand tracking for consistent accuracy.
Deepening our understanding of WebRTC and how to integrate AI-powered features into video calls.

What's Next for UnifyConnect

Expanding Sign Language Support – Training our model on more gestures and different sign languages.
Improving AI Accuracy – Refining our CNN-LSTM model for better precision.
Mobile Support – Optimizing the app for use on mobile devices.
Enhancing User Experience – Adding customization options for captions and accessibility settings.

We believe this project has the potential to make communication more inclusive and accessible for all! 🚀