flatline

Conceptualized, designed, and birthed in 33 hours.

Touch four pulse sensors and your heartbeats become a live four-piece electronic band in the browser. Each instrument follows its own pulse. Anyone can make music, and it's more fun with friends!

Inspiration

Our goal was to create something that would feel just at home in an interactive Museum of Science demo as an interesting music festival artifact, or on a musician's desk. Taking inspiration from the natural rhythms that the human body produces, we set out to build an instrument that could produce music collaboratively and in real time that also gave you a unique feel for the moment.

What it does

flatline turns four people's heartbeats into one live electronic track. Each person puts a finger on a pulse sensor. Their heartbeat becomes one voice of a four-piece band in the browser: drums, bass, lead and pad. Each voice follows its own pulse, and the tempo based on the heart rate of the user. When the hearts sync up, so does the music. Anyone can play it, it needs no musical skill, and it works best with friends. The only creative control is the genre kit: French house, UK garage, ambient or Russian hardbass, and more incoming. The hearts do the rest.

How we built it

Hardware: Four MAX30102 pulse sensors connect to a Raspberry Pi Zero 2 W. We didn't have a multiplexer so we wired straight into the Pi on different pins. Each sensor gets its own bus: one hardware bus and three software i2c-gpio buses. A small OLED display shows pulses with some fun animations, as well as genre kit switching. A lil bit of all-nighter Onshape magic and we get a sleek enclosure that houses these components, along with a 2.5in 5w stereo speaker system. Pi: A Python reader samples at a true 50 Hz, with one thread per sensor. It sends ir0,ir1,ir2,ir3 lines over USB gadget serial, so one cable powers the Pi and carries the data. A missing sensor drops to 0 and is picked up again within about 2 s of being plugged back in. We also added an OLED display, tunable LED power over SIGHUP, and a live pulse-quality meter that runs over ssh. Browser: The browser reads the serial data with Web Serial and does all the sound synthesis. A beat detector with a bad-read filter holds the BPM through shake, drift and finger landing. Each voice evolves from its own signal history and a shared musical grid. Every genre has its own tempo, key, chord-aware bass, drum kit, lead sound (talkbox, chopped vocals, supersaw) and stabs. Tooling: A Simulate mode needs no hardware that a Node script replays through the realbeat detector. Four .mjs test suites cover the audio, fluid and stage code.

Challenges we ran into

The first issue was ideation. Each teammate had a really distinct vision for what they wanted to do with a variety of different interests spanning the teammates' majors: mechanical engineering, computer science, math, and ECE. Ultimately, we arrived at an idea that we thought would be interesting to demo and fun to play with while going through the tedious development process. Initially, we wanted the system to be based on a fingerprint sensor, which we saw in the hackathon parts list. However, since there was only one sensor available, it was not found, and we had to pivot to a different kind of biometric signature. The other issue was the shortage of compute boards. We initially wanted the system to run on more powerful hardware and serve as a standalone device, but given the lack of availability, we shifted to a pi zero instead of a pi 5 and came up with a way to create a unique UI UX experience in the browser.

What we learned

We had a large skill exchange. One of our teammates was using Git for the first time, another had little to no experience doing hardware development or 3D printing, and a third was completely new to the concept of how a synthesizer works. Each teammate therefore also had to be a teacher giving tips to each other while we all worked in parallel and trying to stay patient while under the pressure of a deadline. We collectively learned a lot about UI UX and got introduced to a completely new library, tone.js, that we ended up using to produce our synthesizer sounds

Accomplishments that we're proud of

We're super proud of the web UI that we were able to create on the web interface as well as the fun dancing animation on the OLED screen that sits on the device. The graphics were inspired by ferrofluid sound visualizers, and we think we were able to replicate the cool spiky bouncing effect of the fluid in our web interface. After pulling an all-nighter developing the project, we were super happy to see our roommates come into the living room and want to play with the instrument that we had just made, and ultimately were glad to see them smile and dance when the music came on.

What's next for flatline

More genres, improved accuracy, and potentially integrating even more biometric sensors to create increased personalization and a more unique and immersive experience for the user. Migration to more powerful compute, allowing for standalone system.

Built With

Share this project:

Updates

Submission history