Myo is an amazing device, detecting natural arm and hand gestures and serving them as an input to devices. Myo is already used as a game controller for the games that supports it or with the hacks that are specifically built for it. We wanted to extend Myo's use cases even more.
What it does
AirControl is a hack for Android that enables Myo gestures on the phone UI and games.
How we built it
AirControl is designed to be as general as possible. It is not bound to a single app or a few apps. You can use AirControl to navigate through your home screen and all of the apps that respect accessibility. Plus you can play games that use touchscreen gestures.
Challenges we ran into
Manipulating touch gestures on a typical real device is not allowed due to security concerns. To make this hack happen, we used a rooted Android phone, which lets us send gesture events directly to the operating system.
At first, we wanted AirControl to control an Android Wear smartwatch, as navigating through an Android Wear device could be frustrating due to its small screen. Also, it is nearly impossible to use the device when your other hand is not free. But having a root access on a Android Wear smartwatch is harder due to hardware reasons. We didn't have the necessary equipment to gain root access on a Moto 360, so we gave up on that.
What we learned
Every hardware has its limitations and no matter how easy its API is to use, you need to figure out your own way to make sense of the input data and trigger the action that you want.
What's next for AirControl
It would be awesome to control the input of more sensors in Android devices.