OkMuse

Inspiration

Did you know that a few hundreds of years ago nobody would have even thought about performing on stage by heart? More than that, it was considered to be a sign of arrogance. Our Google Glass (TM) app idea is inspired by the struggles of modern musicians. We're having them too - being pianists and playing in bands.

With the rise of classical school of the late 19th century, everything has changed. Talented teachers started encouraging their students to memorize the pieces exactly as the composers had intended. Till now, pianists struggle with that (nobody can escape a situation when memory fails, even the most talented performers). If you choose to play safe and take someone responsible for sheet turning with you, you need to be sure that the person knows exactly when and how to turn the page. In many cases, this just adds up to stress.

In addition to that, the amounts of paper used are huge. Only during Jan-Nov 2015, our ensemble has printed 140 602 music sheets. That amount of paper roughly equals to two trees, and the weight of a pile of paper of this size would equal the weight of an adult kodiak bear.

Google Glass seemed to be a good base for this problem's solution; that's why we created OkMuse.

What it does

OkMuse follows your performance by listening to it and shows you the current bar as well as the next bar to play. It's created more for reference purposes, but it is also possible to read sheet music with it. More than that, thanks to the compact size of Google Glass, you are able to take your full repertoire with you practically anywhere.

How we built it

OkMuse uses a variety of services to connect different parties. Its core is a Node.JS server hosted on Microsoft Azure as a web application which listens to HTTP GET requests sent by SampleSumo's MusicFollower API (which is listening to the performance), translates it into meaningful information, and passes it on to other interested parties via Socket.io. The music comes from MuseScore, which conveniently hosts its music as pairs of MIDI files and PDF sheets, allowing concurrent following of the performance.

Challenges we ran into

There was only one Google Glass to work with, but luckily the demand was rather low.
The one available Google Glass is a prototype, which means extra hassle and buggy performance (especially with connectivity)
Socket.io was very difficult to implement on the Google Glass, because there isn't a standardized Android client for it, although it is arguably the best option for such realtime connectivity over Web Sockets.
The development workstation in use was unfortunately running a beta version of OSX El Capitan, which basically rendered 90% of Android development tools and applications useless.

Accomplishments that we're proud of

Node.JS server works beautifully on Azure, getting and broadcasting messages from MusicFollower swiftly.
The white-on-black music sheets look futuristic and cool on the Google Glass (plus they're more readable than the traditional black on white)

What we learned

Azure + Node.JS = PROFIT
Working on new prototype hardware is challenging
The Glass microphone itself is not of an acceptable quality for OkMuse's purposes yet
Apple beta programs are not worth it.
Android could use better web socket options
Java should really get rid of that final semicolon.
Google Glass View design and coding is streamlined and sleek

What's next for OkMuse

Proper client application on the Google Glass (possibly switching to the Google Mirror API completely, as opposed to native Glass apps)
Staying tuned for updates on the new generation of Google Glass, hoping it would bring better built-in microphones and eliminate connectivity issues
Getting rid of the fugly watermark on our promo video
Discussion with sheet music providers for long-term cooperation in case the project becomes commercial