Voicify

🎯 Our Goal:

We aim to assist individuals who are Deaf or hard of hearing by enhancing communication through ASL recognition, rather than relying solely on text-to-speech. Sign language is a powerful form of self-expression, culture, and identity—and it deserves to be preserved and empowered through modern technology.

What it does

Voicify translates American Sign Language into real-time visual and text-to-speech outputs using computer vision and machine learning. By capturing hand movements through a webcam, our system identifies both individual letters and full words in ASL, providing instant feedback on what’s being signed. This allows users to communicate seamlessly without relying on speech or typed text. Whether it’s for daily conversation, educational purposes, or accessibility, Voicify empowers individuals to express themselves authentically through sign language—bridging the gap between spoken and signed communication.

How we built it

📸 Computer Vision & Machine Learning

MediaPipe for extracting hand landmark data in real-time. Random Forests for recognizing individual ASL letters from landmark features. LSTM Neural Networks for learning temporal patterns in ASL word gestures and improving word-level recognition

🌐 Full-Stack Web Integration

React Frontend, delivers a smooth, intuitive, and accessible user experience. FastAPI Backend – powers the ML inference engine and processes video input. OpenCV, overlaying real-time MediaPipe hand landmarks and predictions directly on an intuitive and sleek UI.

⚙️ Challenges We Faced:

The most difficult challenge was integrating Python-based computer vision and machine learning models into a live React front-end. Our backend team didn't have extensive experience coding in Python and with respective frameworks, therefore rendering real-time MediaPipe/OpenCV outputs seamlessly on a web canvas took a lot of our time. Another obstacle we faced was handling the noisy, low-light, and grainy video data during training for ASL detection. We wanted to make sure our model was as accurate as possible, and improving model generalization across users with different hand sizes, speeds, and signing styles was another main factor that we focused on.

🌱 What We Learned & Proud Of:

In 24 hours, we built a fully functional full-stack application from scratch and gained deep hands-on experience with computer vision, ML model deployment, and frontend/backend integration. We designed with accessibility and inclusivity at the forefront, reinforcing our conviction that technology should empower—not replace—individual expression.

What's next for Voicify

Voicify is just the beginning of redefining how we support and uplift ASL communication. Moving forward, we plan to expand our word recognition model using larger datasets like WLASL and incorporate gesture personalization to better adapt to individual signing styles. Additionally, we’re exploring mobile deployment, enabling users to sign and communicate on the go with their smartphones. Long-term, our vision is to collaborate with the Deaf community to build tools that are not only accessible—but empowering.