In HackUMBC Spring 2016, we discovered the potential that the Leap Motion sensor has as a means to interact with art. We were able to create a very personal connection between human and machine, awing the senses by linking our hand motions with sounds and animations . Inspired from our breakthrough, we decided to up the level, creating a multiplayer vision of the new art form with more control over our sound output and visuals.

Our new project, Leap Band, utilizes the socket server: socket.io, the javascript sdk for Leap Motioon: leap.js, and a flexible javascript library we found for synthesizing sound: wad.js. With these technologies, anyone with a leap motion sensor and a web browser can visit our site and join the band. All of the users of our project can hear each other's instruments and see each other's animations, creating some serious potential for fun jam sessions and a feeling of community among all the leap motion users who have been dreaming of multiplayer games.

We started our project with a vision and an idea of how to get it done. When hacking time came, we were immediately on task. Eric Hebert and Kevin Miller started working on the web socket server, and Cory Ferrier and I (Nat Baylon), worked on the web client and started researching javascript libraries for visuals and tone production. Three hours in, our socket server was basically set up, and a couple hours after that, our first instrument, made by Cory: the drums, were ready to be tested. The moment we saw that we could play our air drums and hear output on Eric's laptop where we were locally hosting the socket server, we knew that we would be able to accelerate forward and really get going with creating instruments. Then came the piano, which I made, the techno instrument by Eric, and the guitar by Kevin. I got inspiration from Kevin to map the space sensed by the leap motion sensor to an array of pitches, thus creating a scale. The last instrument I made is a blues instrument; the left hand plays chords in the blues progression by moving horizontally, and the right hand plays pitches in the blues scale by moving vertically. We reached our end goal very smoothly and comfortably, and we had a fun time.

We are proud of our hack, and hope more people can see the potential of interactive art.

Share this project:
×

Updates