Conductor

Inspiration

I was inspired by the idea of having computer generated music interact with human expression, and working with Leap Motion gave me the idea for a Soundfont-powered web app that controls some aspects of MIDI playback using only the user's hand movements.

What it does

This project plays a number of MIDI channels which interact.A general chord progression guides a chord backdrop, an arpeggiator, a bass, and other instruments which play a number of different melodies (which can be changed using the keyboard). However, hand motions provided to the Leap control the complexity of the chords and melodies, and the intensity of their playback. For instance, a level, closed fist will silence all instruments. Extending an index finger controls the arpeggiated melody, extending a pinky controls the bass, and tilting the hand controls the chords. Raising the hand makes the notes more legato, and staccato if the hand is lowered. Generally, the user can control the intensity of the playing music using only their hand motion.

Change chord progression, top row of keyboard: (q through t) Change arpeggio melody, middle row of keyboard (a through f) Change bass melody, bottom row of keyboard (z through v)

How I built it

I built this almost completely using JavaScript, jQuery, and the Leap Motion JavaScript API. I also used MIDI.js and some instrument soundfonts I had on my computer to make the music.

Challenges I ran into

Interactively creating music is challenging. In this project, most of the instruments don't algorithmically play, but play pre-programmed melodies. Getting all instruments in sync, and to follow one chord progression were challenges I faced.

Accomplishments that I'm proud of

I successfully read hand gestures from the Leap Motion API, to great effect. You can really control the intensity of the music and come up with some cool songs, if you get the hang of it.

What I learned

I learned that algorthmic song creation (which I originally intended to implement) is a deeply challenging problem. Music is so subjective that it pretty much cannot be left up to a computer.

What's next for Conductor

It could be expanded to have more melodies, or some way of inputting custom melodies or chord progressions. It could better implement gesture control to

Built With

• html5
• javascript