-
-
home page
-
text/audioi to ISL
-
Video
-
About
💡 Inspiration Millions of people in India who are deaf or hard of hearing struggle with communication during video calls due to the absence of real-time Indian Sign Language (ISL) translation tools. We were inspired to bridge this accessibility gap by creating a system that can translate spoken language into sign language live during calls — making communication more inclusive, especially in educational, medical, and professional contexts.
⚙️ What it does SilenceSync is a real-time video calling platform that captures live speech, converts it into text, and instantly plays the corresponding ISL animation videos on the receiver’s screen. It empowers deaf users to understand conversations in sign language without requiring a human interpreter.
🛠️ How we built it Frontend: HTML, CSS, JavaScript, and WebRTC for real-time peer-to-peer video calling.
Speech Recognition: JavaScript Web Speech API for converting live audio to text.
Backend: Django to manage video sessions, user logic, and WebSocket integration.
ISL Translation: Pre-recorded .mp4 ISL videos are mapped to individual words in the speech transcript and played dynamically.
🚧 Challenges I ran into Speech-to-text limitations: Background noise and unclear pronunciation caused inaccuracies.
Word-to-sign mapping: Matching real-time text to available ISL videos required careful pre-processing and file naming consistency.
Real-time synchronization: Syncing animations with speech during video calls was tricky and required precise timing and efficient communication over WebSocket.
🏆 Accomplishments that i'm proud of Successfully developed a working prototype that can make real-time conversations accessible to the deaf community.
Achieved real-time ISL video playback synchronized with live speech in a video call.
Built an inclusive tech solution that goes beyond convenience — into social impact.
📚 What we learned How to integrate WebRTC and WebSocket together in a full-stack Django application.
How to deal with speech recognition limitations and improve reliability using preprocessing.
The importance of accessibility in technology and designing with empathy.
🚀 What’s next for SilenceSync: Bridging Voices and Signs Integrating a 3D AI avatar that performs dynamic ISL gestures using NLP and deep learning.
Expanding beyond word-to-video to enable sentence-level translation.
Adding support for multiple Indian languages and dialects.
Making the system compatible with mobile devices for wider reach and usability.
Log in or sign up for Devpost to join the conversation.