Shaka - Air DJ | Devpost

Inspiration

Shaka was inspired by our desire to reimagine how people interact with music. We wanted to create a system that makes DJing and sound control more natural, expressive, and immersive — something that doesn’t rely on sliders or knobs but instead uses the body’s most intuitive tool: the hands. We wanted people to have a fun and creative way of viewing music, and playing around with Shaka does that.

What it does

Shaka – Air DJ turns your webcam into a real-time, gesture-controlled music interface. Using your hands, you can trigger sound effects, control filters, and switch tracks entirely through movement. For example, a Fist triggers a low-pass filter drop, an Open Palm enables reverb, a Shaka gesture resumes playback, and many more. Each gesture is captured and interpreted live through computer vision, creating a seamless, touch-free way to mix and perform music

How we built it

We built Shaka using Python, MediaPipe, OpenCV, Ableton, and WebSockets. MediaPipe was used to detect and track 3D hand landmarks, while OpenCV processed the camera feed and computed gesture positions, directions, and finger states. We then connected Ableton to our Python scripts to alter the audio via gestures.

Challenges we ran into

The biggest challenge we ran into was utilizing Touch Design. We wanted to use Touch Design to provide an interactive visual for our product, but ran into numerous technical difficulties trying to set it up and merge with our Python script.

Accomplishments that we're proud of

We are proud of creating a functional product completely outside of our comfort zones. During this hackathon, we challenged ourselves to take on roles different from what we were used to, giving each team member the opportunity to explore new areas and expand our technical and creative skill sets. Although this approach took more time and required extra effort, it allowed every one of us to gain valuable knowledge and hands-on experience in unfamiliar domains, making the final result even more rewarding.

What we learned

We learned how to build cross-platform, asynchronous pipelines that integrate computer vision and creative coding tools. We also gained deeper insight into gesture tracking, MediaPipe pipelines, OpenCV optimization, and WebSocket protocols. Beyond the technical learning, we discovered how complex it can be to design human-computer interfaces that feel natural and expressive.