Glitch Studio

• 

Inspiration

Glitch Studio started as a hardware experiment built around an old HP ScanJet 3570c flatbed scanner. I liked the idea of treating obsolete technology less like a peripheral and more like an instrument: something unpredictable, tactile, and capable of producing art through experimentation.

That eventually evolved into a browser-based audiovisual playground for manipulating images, motion, effects, and sound in real time.

For the WebMCP Challenge, I wanted to explore a different question: what happens when an AI agent can play that instrument too?

What it does

Glitch Studio is an interactive audiovisual instrument for creating real-time glitch art and sound.

Humans can manipulate the instrument directly through its visual controls. WebMCP adds another interface: natural language.

Instead of requiring an AI agent to hunt visually for UI elements or simulate a long series of clicks, Glitch Studio exposes structured creative capabilities the agent can use directly.

I can give the agent subjective creative direction like:

Turn this into a hostile machine dream. Push the image toward something colder and more mechanical, make the motion unstable, and give the audio a darker, more degraded character. Surprise me.

The agent translates that intent into changes across the instrument's controls. I can then react to the result and continue directing it conversationally.

The result is a collaborative workflow where the human and AI are playing the same creative instrument through different interfaces.

How we built it

Glitch Studio is a web application whose creative controls are exposed to compatible agents through WebMCP.

Rather than making an AI model infer how the application works from pixels and click coordinates, WebMCP provides structured tools with defined capabilities and parameters.

Importantly, the existing interface remains intact. I didn't want to build a chat box that happened to generate glitch art. I wanted to build an actual creative instrument that remains fun for a human to operate while also becoming legible and controllable to an AI agent.

Challenges we ran into

One of the most interesting challenges was deciding how much control to expose and at what level of abstraction.

Software works in parameters. Humans rarely give creative direction that way. We say things like darker, more chaotic, subtle, mechanical, or hypnotic.

The challenge wasn't simply making an agent capable of changing a setting. It was exposing enough structured control that the agent could translate higher-level creative intent into combinations of actions.

There's also an inherent tension between control and experimentation. Glitch Studio is supposed to produce unexpected results. Giving an agent meaningful control without eliminating that sense of discovery became part of the design.

Accomplishments that we're proud of

I'm especially proud that the WebMCP integration feels like part of the instrument rather than an AI feature bolted onto it.

The same application can now be played in two very different ways: directly by a human through its visual interface, or by an agent through structured tools. More importantly, those modes can work together. A human can give subjective creative direction, let the agent interpret it across multiple controls, react to the result, and continue iterating.

The result isn't simply an agent pressing a button on command. It can interpret an idea like "hostile machine dream" and translate that creative intent into actions within the instrument.

And, importantly, Glitch Studio is still weird and fun to play manually.

What we learned

WebMCP changed how I thought about the interface itself.

Web applications have traditionally been designed around humans clicking, typing, dragging, and tapping. Agents can imitate those behaviors, but that doesn't necessarily mean they should have to.

WebMCP lets the visual interface remain optimized for humans while a structured interface exists alongside it for agents.

For creative software, I think that's especially interesting. Natural language doesn't have to replace the instrument. It can become another way of playing it.

What's next for Glitch Studio

I'd like to expand the creative vocabulary available to agents, expose more of Glitch Studio's audiovisual capabilities, and experiment with agents responding to the evolving state of a creative session rather than simply executing individual requests.

More broadly, I'm interested in the idea behind the project: software that humans and agents can operate together, each through the interface that makes the most sense for them.

Glitch Studio is a deliberately strange example of that idea.

But that's kind of the point.

Built With

• cloudflare
• html5
• mediarecorder-api
• openai-sites
• react
• typescript
• vinext
• vite
• web-audio-api
• webmcp