Perfect Pitch

• 

  Web Interface

Inspiration

Classical musicians as we are, we also listen to pop songs, and sometimes we would even like to play pop songs on our own instruments. Without publicly available sheet music, however, we miss the key often, quite frustrating. It would be better if there is a software that could tell what key that line was in.

What it does

Perfect Pitch does what a person with perfect pitch does: it can tell what note was played (or sang). Even better, it hears the whole song and generates the corresponding sheet music for us to play on our own.

How we built it

ML approach: We first cut a song into pieces each containing word or a single sound, then designed an algorithm to recognize what note that sound is. Melodia Algorithm approach: Use the existing algorithm that has been researched for several years to translate audio to midi and then used Musescore to get sheet music in png. Construct React framework to route backend to frontend and vice versa.

Challenges we ran into

 Traditional audio processing approaches were giving low accuracy, as songs could have varying beats. We decided to go for machine-learning based approaches: CNN and LSTM, for recognizing and generating notes.
 The Melodia algorithm works well on some simply structured songs. However, when the songs are complex and fast, it loses its magic. 
 Some of the routings of the front end and back end is tricky. 

Accomplishments that we're proud of

Single notes recognition reached 90% accuracy. For simple songs, the generated music sheets were mostly accurate, catching the melody and most of the notes.

The website worked seamlessly with mp3 upload and conversion.

What we learned

Techniques in audio processing, both traditional signal processing based methods and machine-learning based approach. Optimize website routing efficiency and Melodia algorithm.

What's next for Perfect Pitch

Improving accuracy on more complex audio. Live tracking of music.

Built With

• html5
• javascript
• librosa
• python
• react