Sign Flow

Inspiration

SignFlow was inspired by the communication barriers faced by deaf and hearing individuals. We wanted to explore how AI could help make everyday conversations more accessible and inclusive.

What it does

SignFlow is a web-based prototype that explores real-time sign language recognition and translation, with the goal of converting signs into readable text and spoken communication.

How we built it

We built SignFlow using Next.js for the frontend and FastAPI for the backend, with Python and MediaPipe for hand landmark detection. The system uses a WebSocket connection to support real-time communication between the frontend and the computer vision service.

Challenges we ran into

Our biggest challenge was developing reliable sign recognition, especially understanding hand movements and translating gestures into meaningful communication. Training and improving the model while working under hackathon time constraints was challenging.

Accomplishments that we're proud of

We successfully developed a working web interface and an end-to-end prototype demonstrating our vision for accessible sign language communication. We are proud of turning our idea into something tangible that can be explored and improved.

What we learned

We learned about computer vision, hand landmark detection, real-time communication, and integrating AI components into a web application. We also learned how important diverse datasets, model accuracy, and continuous testing are for sign language recognition.

What's next for SignFlow

Our next goal is to improve recognition accuracy, develop a more reliable sign language model, support continuous signing and full sentences, and expand accessibility features to make SignFlow more practical for everyday communication.

Built With

• and
• fastapi
• mediapipe
• next.js
• python
• react
• typescript
• websockets