Many of our team members are avid musicians. Our service is meant to help people, like us, to transfer an original piece into professional sheet music. After we thought of the idea of music transcription, we went and found the piano on the 2nd floor. We noticed that we wouldn't be able to use a microphone for our transcription technique since the piano was in a bad condition (out of tune, missing notes, etc.). That's when we decided that we would try using a camera to extract the musical data.
What it does
Our hack is a service/script that records musicians while they play piano. This recording is then transferred into sheet music. Most similar services use microphones or MIDI connections to accomplish a similar goal, however, our hack actually uses input from a simple web camera. Our novel approach is a step-up from regular audio detection which can easily be perturbed by external noise or by faulty instruments. The piano we designed it for has missing keys and is horribly out of tune. Luckily for us, our system doesn't need a functional piano to generate the sheet music, only the visual aspect.
How we built it
To build this service, we started by testing our idea to get a better vision of what would be our plan of action. During this testing time, we worked on creating a support for our camera and finding the most convenient angle to place it at. We then moved on with creating both a line of code, and a web service, to support our vision. Throughout this step, we worked to solve problems, while reaming focused on our end goal.
Challenges we ran into
Our team faced many challenges throughout the project. Our first challenge was trying to come up with a solution to our problem (music transcription) in a creative, yet useful way. With the piano at our disposal, it wasn't initially obvious to use a webcam, but we eventually settled on that idea. Many hours were spent tweaking and debugging the code for our main service, the computer vision requiring additional care. Another challenge we faced was when they taped the piano shut overnight (made too much noise)! As for the web design, we ran into many challenges when it came to incorporating images into our web service. Indeed, many of those images (as pretty as they were) were either not the right size, or the right colour.
Accomplishments that we're proud of
As a team, we are proud of being able to keep our motivation constant throughout the project (even at 3 in the morning!). When working on the code, we were proud to see our project come to life when Andrew was able to combine the video to the code (which had the purpose of composing the sheet).
What we learned
As a team, we learned how to work together, thus combining our assets to create a service with the purpose of helping musicians transfer their compositions into music sheets. We learnt various skills in the fields of coding, presentation, web design and problem solving that we can all apply to our future endeavours. Finally, we also learned the importance of organisation, communication and prioritising in order to be effective throughout our creative process.
What's next for Sharp Music
Next, we would like to be able to integrate the service into a web app to allow our service to run publicly. We would also like to add levels of complexity to our sheets to allow musicians to play in different keys and with a wider range of keys.