KinVis

Inspiration

We wanted to add a whole new layer to the entire concept of a music performance.

What it does

We provide an interface that interfaces between a Kinect (could work with other camera inputs), to use gesture-based audio generation (to a MIDI output) and create visualizations. We've currently interfaced with a Mod Duo device, since it allows us immense flexibility in mapping the gestures. However, our MIDI output could work with Bitwig Studio, and all DAWs that allow a MIDI input! We anticipate that our device could be used in conjunction with a Kinect (or similar sensor) and add an exclusive layer to live performances, so that performers could make their shows unique.

How we built it

A large portion of our product is the hardware component, which comprises interfaces connecting Arduino microcontrollers, the Mod Duo and the outputs from the Kinect code. The code was Python based, and tracks 20 joints in the human skeleton.

Challenges we ran into

A major portion of our time was spent understanding the Kinect documentation, or lack thereof!

What we learned

MIDI generation, Control Chain connectors, Kinect integration, tons of hardware stuff.

What's next for KinVis

We plan to expand our prototype with a lot more gestures, and grand visualizations!

Built With

• python-c