We both play piano and occasionally want to play songs we have audio files for, but sheet music, or tools to convert audio to sheet music, are often unavailable or expensive. We decided to create Musidi to help obtain sheet music for the melody of .mp3, .mp4, or .wav files.
What it does
Musidi first creates a .mp3 file from the user's specified YouTube video, extracts the vocals (melody) from the song as a .wav file, then converts the vocals to a .mid file (MIDI) and presents that to the user along with the .wav of the vocals a visual representation of the frequencies in the MIDI. MIDI files can easily be converted to actual sheet music with any number of online tools, such as https://www.onlineconverter.com/midi-to-wav or https://solmire.com/miditosheetmusic/.
How we built it
Most of the code, except for the webpage, is in Python. Wrapped using Flask to connect it to the HTML, the program (1) gets the mp3 from YouTube using the youtubedl library, (2) extracts the vocals using the spleeter module, and (3) converts vocals to .mid using a pretrained CNN with the pypianorolls library.
Challenges we ran into
We had difficulty exporting the vocals and instrumentals to .wav files because that component was originally written for an interactive IPython environment, but we were eventually able to work that out. We could not connect the program to an online website because the pretrained model was too large and some of the necessary versions of the models conflicted (e.g. only python version 3.7 and numpy version 1.6.5 worked), so the webpage only works locally as of now.
Accomplishments that we're proud of
We are proud that the whole project came together and the final MIDI file sounds somewhat like the melodies for the original music.
What we learned
We learned how to use a variety of different Python packages and manipulate music and audio files. This was also our first time using Flask to connect Python to a webpage.
What's next for Musidi
The closest step for Musidi is transcribing lyrics to the song, which we were able to do on one machine but were not yet able to connect to the webpage. In the future, we should be able to connect it to a website (we have musidi.tech but nothing is connected to it) and put it online with a nice-looking user interface.