Inspiration
Sounded more interesting than writing a vim plugin.
What it does
Sheety reads sheet music. You can play the sheet music back through the use of the "magic wand".
How we built it
Sleep deprivation.
Challenges we ran into
Identifying locations of individual notes on a staff. Identifying the pitch of a note on a staff. Identifying the length of a note on a staff. Tracking our "magic wand" and matching it's location against the staff. Identifying the clef. Identifying the key signature. Sending video data recorded from a browser to our back-end in a way that doesn't butcher the frame-rate or cause artifacts. Playing individual notes on the front-end in such a way that a piece could be played forwards or backwards.
Accomplishments that we're proud of
Writing the character-cutter-outer by hand. Getting the machine learning working in Google Cloud. Tracking the "magic wand".
What we learned
To not trust the written web standard. It was not as robust as expected for media streaming due to implementation differences between standard and reality.
What's next for Sheety
It would be possible put all the individual pieces together to make the intended application pipeline fully functional if we could find a solution for high-quality media streaming or rewriting our character-cutter-outer to run in javascript so that we don't need to stream out video.
Log in or sign up for Devpost to join the conversation.