MURMUR

Turn your Twitch audience into a living orchestra. Viewers contribute real sounds, shape the music through chat, and create a one-of-a-kind audiovisual performance together.

Comment

Your noise becomes music.

A whistle. Two claps. An awkward little hum.

Not exactly the ingredients you'd expect for an orchestra.

MURMUR is a live musical instrument for Twitch that turns these tiny audience recordings into music. Viewers supply the sounds, the streamer conducts, and chat helps decide where the performance goes.

The result is something you can hear, influence and watch take shape through the Living Record, an audiovisual artwork built around the sounds people contribute.

You don't need musical talent to participate. In fact, the imperfect recordings are often the most interesting ones.

I wanted Twitch chat to make something

I've always enjoyed streams where the audience throws the original plan completely off course. Chat votes for something ridiculous, someone suggests an unexpected idea, and the streamer runs with it.

That unpredictability is a big part of Twitch's appeal to me.

But I kept thinking about how quickly those interactions disappear. A message scrolls away. A sound alert lasts three seconds. Everyone laughs, and the stream moves on.

I wondered what would happen if those moments could leave something behind.

What if the audience didn't just choose what happened next, but supplied the actual material?

Not predefined sound effects. Not a generated song with someone's username attached to it.

Their own voices, whistles, laughs and rhythms.

That's where MURMUR started.

So, what actually happens?

  1. A viewer contributes. They join with Twitch and record a short human sound. A whistle, a hum, a clap, whatever they're comfortable sharing.

  2. The streamer listens first. Recordings enter a private moderation queue. Nothing goes into the public performance without approval.

  3. MURMUR finds the music. The composition engine analyses the approved audio and develops melodies, rhythms and textures from the recordings themselves.

  4. Chat joins the conducting. Viewers vote for musical directions such as !vote playful or !vote cinematic. The streamer applies the winning direction and controls tempo, balance and individual contributions.

  5. The music becomes visible. The Living Record evolves with the arrangement, giving the different contributions a visual presence.

There's also a feature called Impossible Solo. It brings an individual recording forward so you can hear the original sound, its musical transformation and its place in the larger arrangement.

I particularly wanted that moment. Without it, you might hear a pleasant piece of music and never realise that the melody began as someone's two-second whistle.

The whistle incident

One of the first real Twitch-connected rehearsals was a bit of a disaster.

I recorded a whistle lasting just over two seconds. MURMUR accepted it, analysed it and played the resulting composition.

The application didn't crash. The tests were passing. The playback report looked perfectly respectable.

But the music sounded wrong.

When we inspected the performance, we found this:

78 rhythm events.

1 texture event.

0 pitched musical events.

From a whistle.

We'd managed to take one of the most naturally melodic sounds a person can make and classify it as percussion.

It was funny for about five seconds. Then I realised how serious the problem was.

With OpenAI Codex, I traced the issue through the audio analysis. The analyser was missing important pitched fragments in recordings with pauses and changing notes. We changed how it examined active sections, added regression tests and compared the new arrangement against the original recording.

The difference was audible.

That experience changed how I approached the rest of the project. I stopped treating a successful test or a completed composition as proof that something was good.

I needed to actually want to listen to it.

Building MURMUR with Codex

Codex was part of the entire development process, not something I brought in to finish a few tasks.

The application uses JavaScript, Node.js, browser-based audio processing and rendering, Twitch APIs, Docker and Playwright. Codex helped me develop the recording flow, moderation, composition engine, Studio controls, Living Record visuals, automated tests and deployment tools.

The most interesting part was learning how to work with it on problems that don't have an obvious correct answer.

A failing API request gives you an error to investigate.

A musical arrangement that feels dull doesn't.

I had to listen closely enough to explain what bothered me. Was a phrase repeating too often? Did the transition feel abrupt? Was the Living Record doing something visually impressive but musically meaningless?

Sometimes I'd ask Codex to improve a sequence, review the result and decide the previous version had more character. We would go back, compare the outputs and try a different approach.

The engineering had its own challenges. Twitch authentication needed to survive token expiry and restarts. Viewer audio had to remain isolated from the public broadcast until approved. The container needed persistent storage without exposing private recordings or credentials.

At one point I was debugging OAuth token renewal while trying to build an orchestra out of whistles and handclaps.

Software development has a peculiar sense of humour.

What surprised me most

Making sound was easy. Making someone care about the sound was harder.

Automated tests could tell me that the engine produced the expected events, that recordings stayed behind moderation, or that a Twitch vote reached Studio.

They couldn't tell me whether the music had any personality.

The same was true of the interface. More controls didn't make Studio a better instrument. I wanted a streamer to spend less time managing settings and more time responding to the audience.

And the visuals couldn't just look impressive in screenshots. They needed to feel connected to the people and sounds behind the performance.

The project taught me to take subjective feedback seriously without abandoning technical discipline. Both mattered, and neither could replace the other.

Where I'd like to take it

I'd love to explore Channel Points and Bits-powered musical moments, more expressive conducting, and ways for different Twitch communities to create performances together.

There's still a lot to discover about what makes an arrangement feel musical when its ingredients are unpredictable human recordings.

But the experience I'm chasing is quite simple.

Someone joins a stream and records a slightly silly sound. They forget about it for a minute.

Then they hear it again.

Only now it's part of a melody. Someone else's rhythm is underneath it. The Living Record is changing with the music, and chat is deciding what happens next.

They recognise their little contribution in the middle of all that.

Wait, that's mine.

That's the moment I built MURMUR for.

Built With

Share this project:

Updates

Submission history