Palmo

An AI-powered app that teaches sign language through real-time feedback.

Comment

Inspiration

Communication is key! Everyone deserves to be heard. We're gamifying ASL to ensure no one is left unheard.

What it does

Palmo is an interactive platform that gamifies ASL learning through bite-sized, accessible lessons. Users progress by completing multiple-choice quizzes and active signing challenges. By utilizing camera-based motion recognition, the app provides real-time feedback on hand gestures, allowing users to earn high scores as they master new signs.

How we built it

Frontend: React for a responsive, interactive user interface. Backend: FastAPI for high-speed asynchronous processing and SQLite for efficient data management. AI Engine: OpenRouter and Google Gemini to handle real-time sign language translation and camera-based feedback.

Challenges we ran into

  1. Lack of Direction ** As first time hackers, we had a hard time deciding what project we were going to decide on. We're thankful there are so many mentors who were able to set us in a clear direction.
  2. Lack of Sleep ** We pushed past our usual limits, working through the night to meet our deadlines!

Accomplishments that we're proud of

  • Accessibility Focus: We are proud to have created a tool that addresses a real-world gap in healthcare and wellness for the Deaf and hard-of-hearing community.
  • End-to-End Development: As new hackers, taking a project from a blank page to a fully integrated React and FastAPI application in a single weekend is our biggest win.

What we learned

  • The Power of Mentorship: We learned that asking for help early can save hours of indecision.
  • Rapid Prototyping: This experience taught us how to prioritize core features (the "MVP") under a tight deadline

What's next for Palmo

  • More Features: We have many goals that we would like to accomplish even after the Hackathon is over!
  • UI/UX Testing: Refining the UI/UX design phase to improve accessibility and user flow.

Built With

Share this project:

Updates