SignBridge

Inspiration

SignBridge started with a simple observation: most tools for the Deaf community require users to look down at a phone. In conversations, that breaks eye contact and disconnects people.

We wanted to change that. Instead of asking users to adapt to technology, we designed something that fits naturally into how people communicate. By using AR glasses, the conversation stays in front of you, so you never have to look away.

The goal wasn’t just accessibility, it was to create something that feels seamless, empowering, and natural to use in everyday life.

What it does

SignBridge translates spoken conversations into sign language in real time using a connected system.

The user wears AR glasses while a mobile app listens to nearby speech. The app converts speech into text instantly and sends it to the cloud. From there, the AR glasses receive the information and display it directly in the user’s view.

A 3D hand visualizes the message through fingerspelling, while supporting text appears alongside it for clarity. Everything happens live, so the user can follow the conversation without looking away or breaking eye contact.

The result is a seamless experience where communication happens naturally, right in front of you.

How we built it

SignBridge works as a connected system between AR glasses, a mobile app, and the cloud.

The AR experience runs on Snap Spectacles, where a 3D hand shows real-time sign language directly in the user’s view, along with supporting text.

The mobile app, built in React Native, handles speech input. It listens to conversations, turns speech into text, and uses AI to create summaries and useful information like notes or calendar events.

We used Firebase to connect everything in real time. As soon as speech is captured on the phone, it’s sent to the cloud and instantly displayed on the glasses.

Challenges we ran into

The biggest challenge was speed. Everything needed to happen instantly — from speech to translation — or the experience would feel broken.

We also ran into technical challenges with 3D. Our first idea was to use a fully animated hand model, but it was too heavy and slow. We had to rethink the approach and build a lighter, more efficient version from scratch.

Another issue was stability. At times, the system would replay old conversations or UI elements would overlap and flicker. We fixed this by separating parts of the system and making each one more independent and reliable.

Accomplishments that we're proud of

We are senior UX Design students and this was our first time building for AR in Lens Studio. We are proud that we built a fully working real-time prototype in just 36 hours. What started as an idea became a complete system connecting AR glasses, a mobile app, and the cloud.

We’re also proud of how seamless the experience feels. Instead of creating just another tool, we designed something that keeps people present in conversations and restores eye contact.

Another big accomplishment was simplifying complex technology. From real-time translation to AI-powered summaries, we made advanced systems feel intuitive and easy to use.

What we learned

We learned that in accessibility, speed really matters. Even small delays can break the experience, so everything needs to feel instant.

We also learned how important it is to simplify. Breaking the system into smaller parts made everything more stable and easier to use.

Finally, we saw how powerful AI can be as a collaborator. It helped us move faster and focus on the decisions that really matter, especially in a short timeframe.

What's next for SignBridge

This is just the beginning. Right now, SignBridge focuses on fingerspelling, but the next step is expanding to full sign language translation with more natural and expressive communication.

We also want to improve accuracy and speed, making the experience even more reliable in real-world situations like classrooms, meetings, and public spaces.

Long term, the vision is to support communication in every direction creating a more inclusive and connected world.