MAX/MSP the flagship product of Cycling74 sports a slick dataflow process for hooking up multimedia elements; everything from sound generation to video manipulation. MAX/MSP has been around for a number of years and is a favorite among transmedia artists.

Fast forward to the present... we now have an amazing technology - Leap Motion. It captures hand motion with an unprecedented level of fidelity at a staggering refresh rate (over 100Hz). Better yet, it's SDK has bindings in a number of languages including JavaScript, C#, Unity3D, C++, Java, Python, and Objective-C.

We wanted to bridge these two worlds. The Solution... Leap Motion streams out over websockets all the hand motion and position data as JSON. We use NodeJS to capture this data, listen to and filter the specific kind of data that we want (be it the hands, movement of individual fingers, palm orientation, gestures, etc.). We convert this information as it streams form LEAP into vectors that MAX/MSP likes to parse and push it to MAX.

Now MAX/MSP gets to do it's magic. It breaks out the data into individual components (see the first Screen Shot). The diagram with

Because of the limited time in this hackathon, we decided to make a Theremin in MAX/MSP powered by Leap Motion. In addition to pitch (sound frequency) and volume level, we also vary the color mood of the Theremin by hand movement. The Red color component reflects Left/Right movement, Green - Up/Down movement, and Blue - Forward/Backward.

To make it all the more fun, we do a live video capture and move the video by the touchless waving of the hand!

Built With

Share this project: