Virtual Drums

• 

  Basic Setup while Debugging

• 

  KV-260 Board

• 

  AMD Viavado

Inspiration

We really just wanted to work with SystemVerilog and play around with the AMD KV-260 FPGA board. We thought that building a set of "air drums" would be a super fun way to test it out and see what the board could do!

What it does

Virtual Drums is a wearable, hardware-accelerated MIDI drum kit. By strapping sensors to your wrists, you can play a full drum set in thin air. The system tracks your hand’s spatial angle to determine which drum you hit (e.g., Snare, Hi-Hat, or Tom-Tom). It continuously monitors your acceleration using a custom physics engine on the AMD KV-260 FPGA to detect the exact microsecond of a “hit” and calculate your swing force, translating it into dynamic MIDI volume.

How we built it

We split the architecture into a Sensor Processing Hub and a Hardware Logic Engine to ensure zero-latency audio performance. -The Sensor Hub (STM32): We wired two MPU-6050 IMUs to an STM32 microcontroller over a shared I2C bus. To avoid data collisions, we pulled the AD0 pin to GND on the left-hand (0x68) and to 3.3V on the right-hand (0x69). The STM32 reads the Gyroscope data to determine the spatial zone of the user's hands. It packages the raw Z-axis acceleration and zone data into a 4-byte payload and sends it over SPI. -The Kria KV260 FPGA: We designed a SystemVerilog hardware design on the AMD Zynq UltraScale+ processor. The FPGA catches the SPI data and feeds it into a custom hit-detection module. -The Audio Bridge: The FPGA sends a trigger signal to a Python script on a local laptop, which converts the serial data into a virtual MIDI device that plays the audio on the GarageBand app.

Challenges we ran into

The Software Setup: Downloading and setting up the AMD Vivado software took hours, and figuring out how to generate the right files to program the board was really confusing at first.

The Hardware Bridge: Making the STM32 talk to the AMD FPGA was really difficult. We spent a lot of time debugging our SPI connection to make sure the data numbers were syncing perfectly and not getting lost in the wires.

The Final Crash: We had the basic hit-detection working beautifully, but when we tried to add the complicated physics math to track the angles of the drumsticks (so we could tell which drum you were hitting), our system broke. We ran out of time to fix the bug before the deadline. On top of that, the Python script we wrote wouldn't work with GarageBand. This final crash resulted in us not being able to record anything, thus not having a video demo at the end. However, we did attach images and a video of cute cats :).

Accomplishments that we're proud of

Even though our final product broke, we are incredibly proud of the core design we pulled off. Our main goal for this hackathon was to learn how to program and interface with an AMD FPGA, and we absolutely crushed that goal!

We built a working hardware pipeline from scratch. We are really proud that we successfully wrote a SystemVerilog "state machine" on the FPGA that actually worked. We got two totally different boards talking to each other perfectly over SPI, read real physical movements from our sensors, and processed that data in hardware. So this was a win in our books!

What we learned

We had a lot of fun and learned so much about AMD FPGAs. We learned how to send high-speed data between boards without losing it. We also learned how to use hardware pins to change sensor addresses so they can share a wire. But most importantly, we got the interfacing between the FPGA and the STM32 to work, and using AMD Viavado software.

What's next for Virtual Drums

First, we want to go back and fix the bug that broke our system. After that, we want to get the computer software working so we can actually hear the drums out loud, and maybe make the math a little simpler so it runs smoother.

Built With

• c
• c++
• systemverilog