Musica

Grading
Home Page
playing a song
sign in
grader (red for wrong, green for correct)

Inspiration

We all quit piano due to various reasons, whether it be moving, a lack of motivation, or it being too difficult. We want people to continue their music journey, and not quit on the way. Furthermore, people are losing their human side through AI, and music can help remedy this.

What it does

Input sheet music, and output a tutorial on how to play the piece!
Grade your own music: send an mp3/audio file of you playing your piece, and a picture of the sheet music. Our algorithm then grades you on how accurate you are.

How we built it

We used React for frontend, and Python for backend, and used FastAPI to connect front and back end. to go from sheet music -> tutorial, we sent a picture from the front end, sent it to the backend using FastAPI, and then used HOMR, a research grade Transformer based Optical Music Recognition (OMR) tool that gives a .musicxml file from sheet music. We then display it on React through React-Piano, and we have an assortment of tools (like tempo increase, a highlighted visualizer, a metronome, and a volume notch) to help users learn interactively.

For grading, we used Spotify's Neural Network Basic-Pitch to convert our .mp3 file to a .MIDI file, and through an algorithm utilizing Dynamic Warping Time, we are able to get an accuracy score that has shown to be pretty accurate, but does pretty poorly on ≥ 2+ notes. We then show the notes that the user made wrong in red, and the correct ones in green.

Challenges we ran into

Getting everything to work visually, as well as connecting the front and back end. It was our first time using FastAPI (or any server), so we had our fair share of difficulties with it. Also, formatting everything with the Piano was hard, but we got through it