HandyBand

user interface of our website
Piano chord function
Creating audio files in FLStudio

Inspiration

We wanted to bring the body into the rhythm—to make music through movement, without needing a physical instrument. HandyBand explores how simple hand shapes and arm gestures can become an approachable way to play.

What it does

HandyBand turns camera-tracked gestures into prerecorded musical sounds. Odysseus combines seven gesture-selected oboe clips with drum hits triggered by downward arm strokes; faster strokes produce louder hits. Piano gives each hand a separate bank of six piano clips, allowing users to combine two musical parts.

How we built it

We use MediaPipe to track hand landmarks and arm joints. Geometric rules identify finger shapes, while relative wrist-to-elbow motion detects drum strokes. Stability checks and per-hand timers control sound triggering.

Challenges we ran into

The central challenge was separating intentional gestures from ordinary movement. We used brief hold confirmation for hand shapes, motion smoothing and thresholds for drum strokes, and independent timing for each hand to reduce accidental triggers while keeping control responsive.

Accomplishments that we're proud of

We brought hand tracking, gesture recognition, and audio playback together into two distinct musical experiences, available through a browser and a packaged Windows application—with no wearable sensors required.

What we learned

Recognizing movement is only part of building a musical interface. Timing, clear gesture mappings, and predictable sound behavior are equally important for making the interaction understandable and enjoyable.

What's next for HandyBand

We would like to improve recognition under varied lighting and hand positions, add clearer visual guidance, and explore customizable gesture-to-sound mappings so users can shape their own musical experience.