Sonic Pick

• 

  Flex strips and ultrasound sensor on the left hand.

• 

  The Arduino wiring.

• 

  Guitar chord classifier confusion matrix.

• 

  David working on the ultrasound distance sensing.

Inspiration

We were watching the Air Guitar World Championships, as one does...

Here's a link to a funny video from last year. Everyone looks really silly. We thought it would be cool if air guitars actually made music, so we decided to make our own, complete with accurate guitar chords and strumming.

What it does

The Sonic Pick emulates a real guitar by sensing hand strumming movement, chord fingering, and hand-to-hand distance for chord pitch. Using the computation power of a Jetson TX2 to achieve real-time Support Vector Machine (SVM) classification for chord recognition, the Sonic Pick detects what the user is trying to "air guitar" and plays the strummed chords!

How we built it

The Sonic Pick hardware consists of two custom-built sensor gloves and a base station. The sensor gloves collect accelerometer, ultrasonic distance, and finger flex data from the hands. This data is pre-processed on a microcontroller and is then fed to the Jetson TX2 for user intent classification and sound synthesis.

Challenges we ran into

At first, we wanted to use the Jetson and a Leapmotion to track the chord patterns through vision and gesture recognition. However, we had trouble with this, probably because we bought black colored gloves -- Whoops! After some fiddling around, we decided to switch to using flex sensors on the fingers instead.

Built With

• accelerometer
• arduino
• machine-learning
• nvidia-jetson
• python