-
A heatmap for session 1
-
Piano roll playback of session one "Rondo Alla Turca"
-
The landing page for Cloud Keys
Inspiration
I was inspired for this project when I wanted a solution for recording improvisations I played on the piano, and viewing how closely I followed a key. I told Jessica the idea and it evolved into the concept of "Cloud Keys".
What it does
Cloud Keys is an interactive user experience both for those who know how to play the piano, and those that don't. Cloud Keys allows you to connect a laptop, raspberry pi, or USB enabled device to continually fetch notes played through MIDI, and send them to the user. The information is then available as a vertical piano roll with audio playback, or a heatmap displaying which notes were pressed the most. This data is split into sessions which are chunks of playback.
How we built it
We connected a raspberry pi to a digital piano using a MIDI cable, and used a nodejs package called "midi" and used socket.io and express to serve the data gathered through the server hardware to the user. The raspberry pi functions as both the web server and the device for connecting MIDI notes.
Challenges we ran into
Just check the white board. But seriously, this was one of the most difficult projects either of us have ever done, and it required so much planning for each individual concept. For example, toggling between old sessions and the socketio data oftentimes led to a segmentation fault in the server, the heat map was upside down and not changing color at all, and using ejs to render HTML templates often ended in nonexistent sessions being offered for playback. There was also another error with the sustain pedal being registered as an A4 because we didn't check what channel the MIDI signal was under.
Accomplishments that we're proud of
Getting a cohesive website together was amazing. At the beginning of the hackathon we were afraid that I (Luke) would take all of the control over the project, but Jessica made an insane contribution by making the landing page, the heatmap page, and making me think everything through before I just copy pasted another snippet of code. Also SSHing through the raspberry pi was crazy, we couldn't get it working until the pi was on my phone hotspot.
What we learned
We learned that to make a good product, you need to plan it out or else the output will be poorly planned. We also got some insight into how servers worked, and all of the various libraries that came with it.
What's next for Cloud Keys
For Cloud Keys, we'd like to add a more comprehensive analytics page (our current one has no numbers, the heatmap is only in one ccolor), and add accounts and profiles to the platform that that multiple users can use it rather than our one raspberry pi. An external server would be especially useful for this. We'd also like to replace our synth sound with a real piano sound and add sustain pedal support.
Log in or sign up for Devpost to join the conversation.