-
Piano
-
DrumKit
Inspiration
Growing up Krishna wanted to be a producer just like his childhood heroes Pierre Bourne and Metro Boomin. He wanted to create beats that him and his friends could enjoy but without the money for instruments or classes, that dream always seemed out of reach. AirBeats was born from the idea that music creation shouldn't be locked behind expensive gear or years of practice but it should instead be as fun as waving your hands in the air.
What it does
AirBeats turns your hands into instruments. With just a webcam, you can play a full drum kit or a piano in mid-air. Each gesture is mapped to a note or sound, and the UI provides live animations and feedback so you can see exactly what notes you are hitting at all times.
How we built it
We used TensorFlow.js and MediaPipe to track hand landmarks in real time. Gestures are classified into drum hits or piano notes, which are then mapped to sound samples with Howler.js. For the frontend, we designed an Apple Music inspired interface with smooth animations and clear feedback for every gesture.
Challenges we ran into
Differentiating subtle hand shapes (like fist vs. open palm). Keeping performance smooth while running live video, AI inference, and audio playback together. Designing a UI that looks polished and professional under hackathon time pressure.
Accomplishments that we're proud of
Building a dual instrument system that works in real time. Creating a sleek, interactive UI that feels like a real working product.
What we learned
We learned how to integrate AI/ML hand tracking into a user-friendly web app, how to design for real time responsiveness, and how to balance functionality with polished design.
What's next for AirBeats
Recording capabilities to play back music/beats that you make. 2 hand capabilities for more accessibility. Adding other instruments such as a guitar. Multi user capabilities so people can collaborate on beats and then also playback editing.
Log in or sign up for Devpost to join the conversation.