-
Homepage, file upload
-
Loading screen while MusicXML is made
-
Choosing instrument
-
Play on your keyboard!
-
The flow of the application
Inspiration
During a late-night conversation with my very musically inclined sister, she mentioned the possibility of me learning to play the piano given the time I spent typing on a keyboard all day. Learning piano is no easy task, however. Learning the piano requires several hours of dedication towards learning how to read sheet music, learning key placements and the subsequent hours of practice that come afterward. At that point, I had an idea. What if I could play any song I want with instructions that are easy for me to understand with a skill that I had been actively using for the better part of the five years.
With Hack The North 2021 rapidly approaching, I thought this would be the best of times to put this idea into motion, and thus SharpTune was born. A dynamic and interactive web application that allows you to play any song right on your computer keyboard!
What it does
Our web application allows you to play nearly any song provided by you using the same instruments! Like a rhythm game with endless possibilities! Simply upload a song, allow us to isolate the different components, and after being provided with the subsequent MIDI and MusicXML file, play the song using our beautifully designed UI.
How we built it
Built with the MERN stack, our web application makes use of various pre-trained machine learning algorithms to dissect the songs you upload into its key components, like the vocals, drums, or piano. After providing us with a MIDI, we then employ more algorithms to convert the MIDI into MusicXML, a format we can parse digitally to allow you to play the song on your keyboard!
Challenges we ran into
The process of converting an audio file, like an .mp3 or .wav into a more machine-readable format like .midi and .mxl was one that we did not anticipate being so difficult. During our work, we discovered that trying to convert an audio file into a midi file was like trying to unscramble an egg. We learnt that this type of software is either proprietary or still in the process of being researched before being made fully open source with an API we could use. We did find web applications that did the job for us but nothing we could integrate into our code directly. Ultimately we were forced to require the user to upload a midi after isolating the components of a song, we did our best to make that process as seamless as possible
Accomplishments that we're proud of
We are very proud of the work we were able to accomplish despite starting the project relatively late compared to other groups. We are also proud of being able to make what we can under the pressure of the circumstances.
What we learned
We learned that before starting a project, we should do more prior research and planning so we don't run into seemingly unsolvable problems during development. We also figured that we should start developing the core feature first before other pages so that we would have more functionality to show.
What's next for Sharp Tune
First and foremost, we want to be able to allow the user to play any song they'd like more seamlessly. This means coming up with a way to convert audio files into midi files, without relying on the user. As enhancements, we hope to introduce better support for more instruments and perhaps introduce new UI changes to improve that support. We would also like to finish developing the keyboard playing screen with the dropping notes like we designed.
Log in or sign up for Devpost to join the conversation.