MAX/MSP the flagship product of Cycling74 sports a slick dataflow process for hooking up multimedia elements; everything from sound generation to video manipulation. MAX/MSP has been around for a number of years and is a favorite among transmedia artists.
We wanted to bridge these two worlds. The Solution... Leap Motion streams out over websockets all the hand motion and position data as JSON. We use NodeJS to capture this data, listen to and filter the specific kind of data that we want (be it the hands, movement of individual fingers, palm orientation, gestures, etc.). We convert this information as it streams form LEAP into vectors that MAX/MSP likes to parse and push it to MAX.
Now MAX/MSP gets to do it's magic. It breaks out the data into individual components (see the first Screen Shot). The diagram with
Because of the limited time in this hackathon, we decided to make a Theremin in MAX/MSP powered by Leap Motion. In addition to pitch (sound frequency) and volume level, we also vary the color mood of the Theremin by hand movement. The Red color component reflects Left/Right movement, Green - Up/Down movement, and Blue - Forward/Backward.
To make it all the more fun, we do a live video capture and move the video by the touchless waving of the hand!