Frame — Design with your agent

Inspiration

Most “AI design” tools hide the work in a chat sidebar. You describe a poster; you get an image back. You don’t co-edit the same artboard.

WebMCP changes that. The page can expose real design actions — add a shape, set a fill, align a selection, export — as tools on the live canvas you’re already looking at. Frame was inspired by that gap: what if ChatGPT and a human shared one Figma-like board, with the agent mutating the same objects you see?

What it does

Frame is an agent-native design canvas:

Add rectangles, ellipses, and text on a shared artboard
Select, move, restyle, align, duplicate, reorder, and delete objects
Inspect layers and properties in a human UI that works without any agent
Let ChatGPT call WebMCP site tools that hit the same store as your clicks
Watch an activity feed and an Agent tools panel so judges can see what the agent can do
Run ?demo=1 for an automatic happy-path walkthrough (optional delay for screencasts)
Export the artboard as PNG

You steer in plain English; Frame’s tools update the canvas in real time. Destructive actions like delete/clear are not marked read-only, so the browser can treat them as consequential.

How we built it

Vite + React + TypeScript SPA for a fast static deploy
Zustand as the single source of truth for both the React UI and every WebMCP execute handler
WebMCP via document.modelContext.registerTool (with navigator.modelContext fallback), feature-detected on the top-level page (no iframes)
Canvas rendered as a live artboard with layers + property panel
Judge-facing Agent tools panel + activity feed
MIT-licensed, no backend or secrets — works on GitHub Pages
Built with Codex

Challenges we ran into

Same-state collaboration — agent tools and mouse edits must never drift. Both paths go through one store.
Read vs write tools — layout suggestions and state inspection are read-only; delete/clear/export side effects are explicit so confirmation policies can apply.
Demo without a live agent — we added a ?demo=1 path so the happy path is visible even when WebMCP isn’t available in the recorder’s browser, while still registering real tools for ChatGPT’s in-app browser.
Keeping it product-shaped — not a bare registerTool PoC: toolbar, layers, properties, export, and a coherent visual system.

Accomplishments that we're proud of

A complete co-design loop on one artboard: create → style → align → export
Non-trivial WebMCP surface (create/update/align/reorder/export), not a single toy tool
Human-first UI that stays fully usable if WebMCP is missing
A second challenge submission that explores a different WebMCP strength than shopping or booking: creative co-presence

What we learned

WebMCP shines when the agent and the human must see the same thing. A canvas is the clearest proof: when set_fill runs, you watch the rectangle change. That shared visual ground is harder to fake with a headless API MCP server — and it’s exactly what Frame is for.