SignBridge Lite: Empowering Silent Voices

Bridging the gap between silence and speech — a real-time sign language translator and inclusive social hub with AI gesture training and accessible interaction.

Comment

Inspiration

Our inspiration was born from a desire to bridge the communication gap faced by millions in the Deaf and hard-of-hearing community. We imagined a world where technology could make conversations more fluid and inclusive. This led to the creation of SignBridge Lite, a tool designed not just to translate, but to connect people by empowering every voice, signed or spoken.

How We Built It

SignBridge Lite is a progressive web app that functions as both a real-time sign language translator and an inclusive social hub.

At its core, the app uses a user's webcam to capture hand movements. We process this video stream in real-time using Google's MediaPipe to detect hand landmarks with high precision. Instead of a heavy machine learning model, we used Fingerpose, a lightweight, rule-based gesture description library. This allowed us to define common signs (like "Hello," "Thanks," "Help") and have them recognized instantly, right in the browser.

The standout feature is the custom gesture training system. Users can define their own signs, record themselves performing the gesture, and train the application to recognize it. This personalizes the experience and makes the tool adaptable to individual signing styles.

All of this is wrapped in a modern frontend built with React and Vite, and styled with Tailwind CSS. For the backend, we chose Supabase to manage user authentication, social profiles, and to store the posts, comments, and custom gesture data for our social feed.

The result is a seamless experience where users can:

  • Translate sign language to text and speech in real-time.
  • Create a profile and interact with a community through posts, images, and videos.
  • Train the AI to understand their unique, custom signs.
  • Customize their experience with a full suite of accessibility settings.

Challenges We Faced

Our journey wasn't without its challenges.

  • Defining Accurate Gestures: Writing robust gesture rules for Fingerpose that work across different hand shapes, camera angles, and lighting conditions was a significant challenge.
  • Real-Time Performance: Ensuring the entire pipeline—from camera capture to landmark detection to gesture recognition and UI updates—ran smoothly without lag was critical for a good user experience.
  • Inclusive UI/UX: Designing an interface that is intuitive for everyone, especially considering the accessibility needs of our target audience, required thoughtful iteration on every component, from the live captions to the settings drawer.

What We Learned

This project was a tremendous learning experience. We learned that accessibility is not an afterthought; it must be woven into the fabric of a project from day one. We discovered the power of lightweight, in-browser AI with tools like MediaPipe and Fingerpose, which make powerful features accessible without relying on heavy cloud infrastructure. Most importantly, we learned that building an empathetic product requires a deep understanding of the user's daily reality.

What's Next for SignBridge Lite

We are excited about the future! Our roadmap includes:

  • Expanding the default sign dictionary to include more languages (ASL, BSL, etc.).
  • Developing a full PWA (Progressive Web App) for offline capabilities and mobile home-screen access.
  • Implementing a voice-to-sign feature for fully bidirectional communication.
  • Allowing users to share their custom-trained gesture libraries with the community.

Built With

  • bolt.new
Share this project:

Updates