Sushi: A DAW Built in Code, Controlled Through WebMCP

Inspiration

Sushi began with a compatibility problem disguised as a music problem.

My collaborator Avery and I both love making music, and we share a lot of the same taste, but we approach the process from almost opposite directions. Avery is comfortable in traditional DAWs. He uses Logic Pro, has experience with FL Studio and BandLab, and understands music through tracks, effects, timelines, and familiar production controls.

I have used those tools, but they have never been the way I naturally want to make music. I am much more interested in Strudel and TidalCycles: writing patterns as code, changing them live, and treating a composition as something programmable.

That left us without a good place to meet.

When Avery sent me a Logic project, I could not simply open it on Linux. Even if we were using DAWs on the same platform, proprietary project formats and different software would still make collaboration awkward. In the other direction, I could send him a Strudel composition as a few lines of text, but unless he learned the syntax, there was not much he could do with it.

We wanted a workspace where neither of us had to abandon the way we preferred to work. Avery should be able to interact with music through a visual interface. I should be able to work directly in Strudel. An agent should be able to participate without pretending to be a person clicking through a DAW.

That shared workspace became Sushi.

What it does

Sushi is a browser-based, code-native DAW powered by Strudel and its TidalCycles-inspired pattern language.

Underneath the interface, the composition is still code. That makes a Sushi project lightweight, inspectable, programmable, and easy to share. On top of that code, Sushi provides a more familiar music-production interface for arranging tracks, selecting sounds, adjusting effects, and shaping a composition without requiring the user to understand Strudel syntax.

The important part is that these are not two separate modes.

When someone changes a supported property in the interface, the Strudel source changes with it. When someone edits the source, Sushi reflects that change back into the interface. A traditional producer, a live coder, and an agent can therefore manipulate the same underlying composition instead of passing increasingly lossy translations between different tools.

WebMCP adds the third side of that collaboration.

An agent can inspect the composition and its current playback state directly in the page, modify or write music, validate the result, and observe the updated source and interface without requiring the user to copy code back and forth between a chat window and the editor. When an edit fails Strudel validation, the agent receives diagnostics from the running environment and can correct its work.

This feedback loop makes a major difference. The agent is not writing music blindly and returning a plausible-looking block of code. It is working inside the actual instrument.

Because the composition itself is programmable, the musical program becomes a high-leverage control surface for both humans and agents. The DAW is built in code, and WebMCP gives the agent access to that code in its real context.

How we built it

Sushi started as a relatively simple Astro application written in TypeScript.

Astro was a pragmatic choice because it is the framework I already enjoy using, but it also gives the project room to grow without splitting everything into separate repositories or deployments. The application, documentation, and other supporting pages can coexist in the same project while the interactive workspace remains focused and lightweight.

Inside that scaffold, we integrated Strudel as the musical runtime and built Sushi’s interface around its source representation. We deliberately avoided treating the interface as a separate DAW model that merely exports Strudel later. Instead, every feature has to respect the same contract:

Anything that happens in the interface must be represented in the code, and anything represented in the supported code must remain visible and editable through the interface.

That constraint shaped nearly every architectural decision in the project. A track cannot exist only as hidden application state. An effect cannot be added through a UI-only shortcut that the source does not understand. The interface must work with the musical program rather than bypassing it.

We then connected the workspace to WebMCP so an agent could participate through that same underlying model. Rather than automating coordinates or simulating mouse clicks, the agent can work with the composition structurally and receive feedback from the running application.

Sushi registers a typed WebMCP surface against the browser’s native model context. Agents can inspect a structured snapshot of the project, read or patch the canonical Strudel source, validate candidate source against the running Strudel engine, control playback, edit tracks and timeline state, and load or inspect composition templates.

Mutating operations carry a base revision and transaction ID, so stale or duplicated actions are rejected instead of silently overwriting human changes. Successful source edits pass through the same validation and source-to-interface synchronization path as human edits, returning structured state, diffs, and diagnostics to the agent.

The result is a shared artifact with three possible collaborators: one person working visually, another working in code, and an agent working through WebMCP.

Challenges we ran into

Performance became a challenge almost immediately.

Strudel is already a mature and highly interactive environment, so Sushi could not merely produce the same output eventually. Editing and playback had to feel responsive enough to function as an instrument. As we introduced interface state and synchronization, we repeatedly found places where our application was doing substantially more work than Strudel itself.

We had to become much more careful about when the composition was evaluated, what caused components to update, and how changes moved through the application. Music software makes performance problems unusually obvious: latency and interruption are not just technical metrics. They directly affect whether the creative process feels usable.

The larger challenge was maintaining synchronization between code and interface.

Almost every feature sounds straightforward until it has to work in both directions. Adding an effect button is easy if it only changes hidden state. It becomes much harder when that action must create valid Strudel, preserve the rest of the user’s source, remain editable as text, and still be understood by the interface after the code changes again.

It was tempting at several points to introduce shortcuts: parallel state, special internal representations, or UI features that only resembled Strudel behavior. Those approaches might have made an individual feature easier to ship, but they would have broken the central promise of the project.

WebMCP introduced another form of synchronization problem: an agent may act on a project while a human is still editing it. We addressed that by requiring mutating operations to carry a base revision and transaction ID. Stale or duplicated actions return a structured conflict instead of silently overwriting newer work.

Over time, we established more reliable patterns for preserving these contracts. Once those patterns became part of the architecture, development became noticeably smoother. I could describe a feature to my coding agent and expect it to extend the existing model rather than inventing another path around Strudel.

Accomplishments that we’re proud of

The accomplishment we care about most is that Sushi stopped being merely a hackathon idea and became something we actually wanted to use.

A large part of building it was the impatient feeling that we needed to finish the next feature so we could stop developing for a moment and start making music with it. Avery and I have already been able to play with the workspace from our different perspectives, which is the original problem Sushi was meant to solve.

We are also proud that the visual editor and code editor are not competing sources of truth. They are two views into the same instrument. That gives Sushi the accessibility of a DAW without giving up the expressive power and portability of Strudel.

The WebMCP integration demonstrates something we find especially exciting: an agent interface does not always need to be a giant catalog of tiny actions mirroring every control in an application. By exposing a coherent, programmable representation, a bounded set of high-level operations can give an agent meaningful control over an entire creative system.

The agent is not clicking blindly through a simulated DAW. It is operating on the actual composition, receiving structured state and validation feedback from the instrument itself.

Most importantly, Sushi makes human-to-human-to-agent collaboration concrete. Avery can work visually, I can work in code, and an agent can help either of us develop the same piece. Nobody has to translate the project into a different format before the next collaborator can participate.

What we learned

We learned that code can be more than the implementation behind a creative tool. It can also be the project format, the collaboration medium, and the interface between humans and agents.

We also learned that immediate feedback is essential for useful agent collaboration. An agent becomes much more capable when it can act inside the real environment, observe whether its work succeeded, and revise it accordingly. Without that loop, generated music code can look correct while failing as soon as someone tries to play it.

Another major lesson was that synchronization has to be treated as an architectural invariant, not a feature added afterward. The moment the interface and source are allowed to disagree, every new capability increases the number of possible inconsistencies. Holding onto a single shared representation made individual features more difficult at first, but it made the project much more coherent as it grew.

We also learned that human-agent collaboration requires explicit coordination. Revision-aware mutations and structured conflicts may sound like implementation details, but they are what allow a person and an agent to work on the same artifact without silently destroying each other’s changes.

Finally, we learned that good agent interfaces are not necessarily the ones with the largest number of tools. Sometimes the most powerful thing an application can expose is the underlying material the user is already working with. In Sushi, that material is the musical program itself.

What’s next for Sushi

We are going to keep building Sushi because we want to keep using it.

The next stage is to expand the visual side of the application until someone can create and arrange a complete track without needing to look at the source at all. At the same time, we want to preserve the freedom that makes Strudel exciting for people who do want to work directly with code.

We will continue improving performance, expanding the sounds and effects available through the interface, strengthening the synchronization model, and adding features based on what we discover while making music ourselves.

Longer term, we want Sushi to become a genuinely collaborative web-based DAW: a place where traditional producers, live coders, beginners, and agents can all work on the same composition without translating it into separate tools or proprietary project formats.

Sushi began as a way for two people with incompatible workflows to finally make music together. WebMCP expands that idea beyond the two of us. It turns the agent into another participant in the workspace—not a chatbot standing outside the creative process, but a collaborator working on the same instrument, through the same underlying composition, alongside everyone else.