Motion Voice

• 

  Project

• 

  Project

Inspiration

Communication shouldn't be a barrier. Millions of Deaf and hard-of-hearing individuals rely on American Sign Language (ASL), yet real-time communication with hearing individuals remains a challenge. We wanted to build a tool that uses vision and AI to enable seamless interaction — hands speak, and machines listen and speak back.

What it does

We developed a real-time ASL-to-speech translator using computer vision, deep learning, and hardware feedback. Our system captures hand gestures through a camera, classifies them using an LSTM model, and outputs both visual and audio feedback — making communication more intuitive and accessible.

How we built it

We built using Media Pipe

Challenges we ran into

1. Not enough time to both train the model and Find better datasets.
2. The datasets is too small and we have to take video to make our own.
3. Original big LCD screen is broken:(
4. want to integrate camera but we couldn't find a affordable embedded camera in person and it has to be shipped.
5. Originally we are planning to use single board computer but since we don't need to use embedded camera, LSTM and python libraries is relatively harder to been built on SBC. Therefore we chose PC and combine with MCU
6. Model Accuracy is not very high ## Accomplishments that we're proud of
7. Sucessfully integrated the core function of AI algorithms and embedded solutions.
8. Learned a lot of things!
9. Integration is not easy but we have managed to done several components together ## What we learned
10. Futher reinforce the AI skill
11. Explored the possibility between software and embedded systems.
12. To manage our time better ## What's next for Motion Voice

Built With

• gemini
• google-text-to-speech
• hardware
• lstm
• mediapipe
• opencv
• python
• pytorch
• stm32