Auxweave

Inspiration

We kept seeing the same problem: AI agents are remarkably good at understanding what we want to create, but much less reliable at understanding where things belong.

Ask an agent to create a poster and it can generate the right headline, images, colors, and shapes—but it may put text on top of text, place an image over important content, or destroy the visual hierarchy.

We wanted to explore a different question:

What if the canvas itself could help an AI agent understand space?

That idea led us to Auxweave.

What it does

Auxweave is a WebMCP-powered creative canvas that lets AI agents design alongside humans.

Instead of exposing the canvas as a collection of arbitrary pixels and coordinates, Auxweave gives agents structured design tools and spatial context.

An agent can create and manipulate:

Text
Images
Shapes
Frames and layouts
Spatial relationships between elements

The canvas maintains awareness of its objects and geometry, allowing agents to compose and refine designs while reducing collisions and incoherent layouts.

The goal is simple: make the web canvas an environment AI agents can actually understand and create within.

How we built it

We built Auxweave as a web-native design environment with WebMCP at the agent interaction layer.

Underneath the interface is a structured scene representation that tracks canvas dimensions, objects, positions, sizes, and relationships. We then built spatial logic around that representation so the agent can work with higher-level design concepts instead of having to manually reason about every pixel.

The architecture separates creative intent from geometric execution.

The agent can express an intention such as:

“Place the title above the hero image with some breathing room.”

The canvas can then determine the appropriate geometry, validate the result, and keep the composition within the canvas.

This makes the system less dependent on any single model or AI client.

Challenges we ran into

The hardest problem was spatial reasoning.

Our first instinct was to let the agent control objects directly through coordinates. That worked for simple actions but quickly became unreliable as designs became more complex.

We discovered that an agent can make individually reasonable decisions that produce a terrible overall composition.

For example, every object might be correctly created, but their combined layout could still contain overlaps, inconsistent spacing, or broken hierarchy.

The solution was to stop treating the canvas as a passive surface and start treating it as an environment with structure, constraints, and state.

Accomplishments that we're proud of

We're proud that Auxweave goes beyond simply asking an AI to generate a design.

We created a system where an external agent can interact with a live creative environment through WebMCP and operate on structured design elements.

We're especially proud of the idea behind the architecture:

The model provides creative intent. The environment provides spatial intelligence.

This means the same canvas can potentially work with different agents without requiring every model to independently learn how to reason about our editor.

What we learned

Our biggest lesson was that better agents aren't always the answer—sometimes the environment needs to become more understandable to the agent.

We learned to think of tools as an interface between an AI's reasoning and the real world. If that interface exposes only low-level operations, the model has to solve too much itself.

Good agent environments should provide:

Structured state
Meaningful abstractions
Clear relationships
Deterministic constraints
Feedback when an action fails

That changed how we think about building AI-native applications.

What's next for Auxweave

Auxweave is only the beginning.

We want to evolve the canvas into a more capable agent-native creative environment where multiple agents can reason about composition, typography, assets, branding, and visual hierarchy while humans remain in control.

We're also interested in making the underlying spatial interface more general-purpose, so that other web-based creative applications can expose their environments to agents in the same way.

Ultimately, we want Auxweave to explore a simple idea:

If AI agents are going to work on the web, the web needs better environments for them to work in.