Soundify

Inspiration

Most of us have learned music theory before, often through sight reading platforms like Sight Reading Factory. However, in loud classrooms setting, these programs are difficult to use due to their reliance on sound. Furthermore, young children may not be apt to understanding music notation and would benefit from an accessible activity to bolster their rhythmic skills.

What it does

The web app generates a rhythm that is illustrated in a scrolling bar which the user must tap out using their hands. The webcam then captures finger movements to detect rhythm accuracy.

How we built it

We used Svelte for the web frontend, and Mediapipe for the hand recognition portion.

Challenges we ran into

We ran into many roadblocks while integrating the computer vision code and webcam display into the Svelte frontend.

Accomplishments that we're proud of

We learned a great deal from designing responsive pages in frontend and linking computer vision with our display.

What we learned

We've bolstered our abilities in designing reactive frontends, as well as taken our previous knowledge on computer vision and iterating on it within a full-stack framework.

What's next for Soundify

We plan to pursue further developments in UI improvement, scalability, and broader functionality, especially in increasing the customizability of the Practice page (difficulty level, web display, etc.).