Duet | Devpost

The studio — full five-instrument arrangement, Musical Contract panel, 14 live tools, and the timeline.
Performance mode — the swapped toolset (7 performance tools), energy, live meters, and a timeline of the entire agent build.
Contract approval
Share

Inspiration

Most people have a tune in their head that never becomes a song. The melody is the easy part. The hard part is everything around it: drums, bass, chords, and knowing how to put them together.

Most AI music tools solve this with a text box: describe a song, get a song. I wanted the musician to stay in charge. You play the idea, you keep the final say, and an AI agent joins as a bandmate, not a ghostwriter.

WebMCP makes that possible. Instead of building a chatbot into a music app, Duet lets the agent you already use work inside the same live studio, with real tools and clear limits.

What it does

Duet is a browser music studio shared by a human and an AI agent.

The human starts by drawing a melody on a piano roll. It lives only in that browser tab; nothing is uploaded. Then they hand the rest to their agent with an ordinary request, for example to keep the notes as they are and build a cinematic synthwave arrangement around them. The agent reads the exact live composition through WebMCP, adds drums, bass and atmospheric chords, shapes the groove, and performs the result.

Every agent action appears immediately in the interface, is attributed in a visible timeline (HUMAN / AGENT / SYSTEM), and can be heard through the browser. The musician can switch back to Compose mode at any point and change whatever they like, and the agent can respond to those changes. Both work on one composition, and the human keeps the creative direction.

Before the agent touches anything, the musician sets a Musical Contract: the melody is locked to the human; the agent may edit drums, bass, pad, keys and the mix; energy is capped; tempo and key can be locked. These terms are live state that every WebMCP tool enforces. If the agent tries to change the locked melody, its tool call pauses and Duet shows the musician the exact proposed change to approve or decline. The agent only proceeds with permission.

When the song is ready, it can be shared. Publishing always pauses for the musician to approve the title, then produces a backend-free remix link with the whole composition encoded in the URL. Anyone who opens it can play the song, remix it, and invite their own agent to join the band.

Duet contains no embedded AI model and needs no AI API key. The user brings a WebMCP-compatible agent, and WebMCP gives it structured, scoped access to the studio.

How I built it

Duet is built around one shared musical state used by both the visual interface and its WebMCP tools. Whether the musician edits a note by hand or the agent writes a drum pattern, both go through the same engine, bump the same version, and land in the same timeline.

The page exposes a small, typed set of tools: read the session, add and remove instruments, edit each instrument, adjust tempo, key and groove, and publish. The toolset is contextual. Adding a drum machine registers drums_edit; removing it withdraws the tool. Compose mode exposes composition tools; entering Performance mode withdraws them and registers a separate set: play, stop, energy, track mix, breakdown. The registration adapter targets Chrome's document.modelContext implementation, feature-detects the navigator.modelContext proposal as a fallback, and was verified against Chrome's real WebMCP runtime on the deployed site.

Playback is phrase-aware rather than a bare loop: patterns run as a four-bar phrase (main, main, variation, fill), notes carry velocity and small timing offsets, and deterministic humanization and swing keep the groove alive without randomness. The audio engine is built on Tone.js with five synthesized instruments, per-voice filter envelopes, 32 presets, kick-driven ducking and a proper master chain. It reads the shared state on every tick, so an agent edit is audible on the very next step.

The key architectural decision was to keep the model outside the product. Duet is the instrument; WebMCP is the collaboration layer that connects it to whichever agent the musician brings. The app is a static React + TypeScript build with no backend, deployed on Vercel.

Challenges I ran into

The hardest part was designing agent tools that are expressive enough for real musical collaboration without exposing one enormous "make a song" command. Each instrument got a single tool with a small set of enumerated operations, strict validation, and structured responses that tell the agent exactly what changed.

Preserving human authorship was a design problem as much as a technical one. The answer became the Musical Contract: constraints the human sets in the interface and the tools enforce, including the pause-and-ask flow for the locked melody. The agent should respond to the musician's ideas, not replace them.

Beyond that: keeping manual edits and agent actions perfectly in sync, accurate musical timing in the browser, and making every agent action understandable and reversible. 116 automated tests cover the tool sets, mode transitions, contract enforcement, sharing and undo.

Accomplishments that I'm proud of

Duet makes human–agent collaboration both visible and audible. Nobody has to imagine what the agent is doing: you watch instruments and tools appear, see every move attributed in the timeline, and hear the arrangement grow.

The Musical Contract turns WebMCP's human-in-the-loop story into something a musician actually wants: not a permissions dialog, but a bandmate asking before stepping on your solo.

And Duet works without a chatbot or a single model tied to it. The same studio can be played by any WebMCP-compatible agent. WebMCP isn't an add-on here; it's what lets the musician and the agent share one creative environment.

Everything you create is remixable and can be passed from person to person, each inviting their own agent to join the band.

What I learned

WebMCP becomes powerful when a website behaves like a shared instrument rather than a collection of forms.

Structured musical operations are far more reliable and expressive than asking an agent to click coordinates or edit raw files, and contextual tools help the agent understand what is possible right now. Making the human's constraints part of the tool contract, rather than hoping the agent behaves, is what made the collaboration feel trustworthy.

Human–agent interaction feels right when the result is immediately visible, reversible, and, in Duet's case, audible.

What's next for Duet

A song mode that arranges the loop into a full two-minute structure (intro, build, drop, breakdown, outro) using the same mute, energy and phrase machinery, so an agent can arrange as well as play. MIDI keyboard input, so the tune in your head goes straight from your fingers to the piano roll. Variation and fill slots for every instrument. And richer real-time improvisation between musician and agent.