Semamore

Talk With Your Hands, Not Your Thumbs.

Comment

This groundbreaking piece of software translates physical 'Semaphore' signalling into an accessible, intuitive messaging experience.

Inspiration

The semaphore communication system

What it does

Allows for seamless, hands-free messaging between virtually anyone by using advanced machine learning algorithms to translate semaphore body signals into readable characters (semamore).

How we built it

We used the Google built library MediaPipe to do body pose estimation, combined with extremely innovative and sophisticated mathematics to estimate the angle of each arm and convert it into a character. Then this is sent in real-time to a cloud based messaging service, allowing for access anywhere.

Challenges we ran into

  • Accounting for camera mirroring
  • Sleep

Accomplishments that we're proud of

  • The name
  • Created something useable

What we learned

  • Creating a full stack web application
  • Assigning tasks based on strengths to each team member

What's next for Semamore - Talk With Your Hands, Not Your Thumbs.

  • Upgrading to more complex physical languages such as BSL to further widen inclusivity and accessibility
  • Integrate useful LLMs (such as Gemini) to provide useful, efficient word auto-complete for users.

Built With

Share this project:

Updates