SignBridge

Inspiration

My inspiration came from working with Muskaan NGO, where I got to spend time around Deaf and specially abled learners. I saw that many education tools are built for the average classroom first, and accessibility is added later. For many Deaf learners, sign language is their first language of thought and communication. Written English is a different language system, with different order, grammar words, and sentence structure. But most tools treat their writing like normal grammar mistakes. That made me think: what if we stop treating Deaf writing as broken English, and start treating it as a bridge between two language systems? That idea became SignBridge.

What it does

SignBridge is an AI literacy bridge for Deaf and sign-first learners. A learner can write a sentence in their natural style, like:

Store I go yesterday.

SignBridge understands the meaning, breaks it into visual concept cards, and teaches how it becomes written English:

I went to the store yesterday.

It includes an adaptive writing tutor, visual concept cards, guided hints, reading practice, PDF/text lessons, fingerspelling practice, accessibility settings, offline fallback lessons, and a teacher view for progress. The goal is not just to correct sentences. The goal is to make the language bridge visible.

How we built it

We built SignBridge as a full-stack web app. The frontend uses React and Vite as a Progressive Web App, so it can run in the browser and work offline with fallback lessons. The backend uses Node.js and Express to connect with Google Gemini. The AI returns structured responses, including the corrected sentence, focus skill, explanation, visual concept cards, practice, and feedback. For fingerspelling, we used MediaPipe hand tracking directly in the browser, so camera video stays on the learner’s device. We also built a local learner profile, teacher dashboard, accessibility controls, and a benchmark to test the AI’s skill diagnosis.

Challenges we ran into

The hardest challenge was making the AI helpful without making it overwhelming. We did not want SignBridge to dump a big grammar explanation on the learner. So the app teaches one useful skill at a time. Another challenge was reliability. AI APIs can be slow or rate-limited, so we added reviewed offline fallback lessons. This keeps the app usable even when the live AI is unavailable. We also had to think carefully about accessibility, privacy, and trust. The app needed to be simple, visual, respectful, and safe for the people it is trying to help.

Accomplishments that we're proud of

We are proud that SignBridge is not just a chatbot. It is a working literacy system with writing, reading, fingerspelling, offline support, accessibility settings, and teacher-visible progress. We are also proud of the Visual Concept Bridge. It turns a sentence into meaning cards and shows how the learner’s idea changes into written English. Another big accomplishment is making the app reliable. SignBridge has structured AI responses, schema validation, offline fallbacks, and a benchmark showing strong skill-diagnosis performance. Most importantly, we are proud that the project is built around respect. It does not treat Deaf learners as broken writers. It treats them as learners crossing between two languages.

What we learned

We learned that accessibility is not a feature to add at the end. It has to shape the whole product. We also learned that good AI education tools should not only give answers. They should understand the learner’s context, explain one step clearly, give practice, and help the learner build confidence. This project also taught us how important it is to design with empathy, especially for learners whose needs are often ignored by mainstream tools.

What's next for SignBridge

Next, we want to test SignBridge with more Deaf learners, teachers, and accessibility-focused organizations. We also want to improve the fingerspelling practice, add more sign-aware reading lessons, support more sign languages, and build better teacher tools for classrooms. The long-term vision is to make SignBridge a personalized literacy bridge for Deaf and sign-first learners anywhere.

Built With

• api
• app
• css
• express.js
• gemini
• google
• html
• javascript
• mediapipe
• node.js
• progressive
• react
• tailwind
• tasks
• vercel
• vision
• vite
• web
• workbox