Draw_generatoe (Codex draws on canvas)

banner
GIF
demo

What it does

Draw_generatoe lets Codex control a live tldraw canvas through WebMCP. Users can ask it to create pages, draw scenes, add shapes and text, search and decribe youtube embed url in canvas ,Mermaid js ,finds non CC wikipedia commons image , arrange objects, and refine designs through natural-language instructions.

For example, Codex can create a welcome screen, draw a car on a road, notice misaligned text, and correct the layout without replacing the user’s existing work.

Inspiration

The idea was inspired by OpenAI’s YouTube video, Math Problems with GPT-4o featuring Imran Khan from Khan Academy, published on May 13, 2024.

Watching someone interact naturally with AI , made me wonder: what if we could do the same thing on a visual canvas?

The original repository had been on my computer since 2025 . With the arrival of WebMCP, I returned to it and added tools that let Codex place, resize, move, and arrange canvas objects. The goal was simple: describe a scene in plain language and watch Codex draw it directly on the canvas.

How we built it

We built the interface with React, Vite, and tldraw. The application exposes structured WebMCP tools that Codex can use to inspect pages and create, move, resize, align, layer, and draw canvas objects.

Each request follows a safe workflow: Codex inspects the current canvas, preserves populated pages, performs bounded mutations, fits the result into view, and visually checks the finished composition. We packaged the client application and deployed it publicly through ChatGPT Sites.

Challenges we ran into

The hardest challenge was translating creative language into exact canvas coordinates, sizes, and layer ordering. Text and shapes can have different measured bounds, so something that looks centered mathematically may still appear misaligned.

We also had to preserve existing pages, manage overlapping objects, verify WebMCP tool schemas, handle tldraw licensing for the deployed domain, and ensure that public client-side configuration did not expose real secrets.

Accomplishments that we're proud of

We created a working bridge between natural language and a live visual canvas. Codex can produce complete multi-object scenes, native shape-based layouts, hand-drawn decorations, mermaid js ...

We are especially proud that it can inspect its own result, identify visual problems such as incorrect layering or alignment, and improve the canvas while leaving unrelated work untouched.

What we learned

We learned that reliable AI drawing requires more than generating shapes. The agent needs structured tools, clear object identities, page awareness, predictable coordinate systems, and visual verification after every meaningful change.

We also learned that small corrective tools—move, align, resize, reorder, and inspect—are just as important as the initial creation tools.