What it does

Codex uses WebMCP to draw, arrange layouts, explain maths, create diagrams, find images, build tables, and run interactive presentations.

Tool vocabulary

Domain Tools
Discovery list_tools describe_tools
Canvas reading capture_canvas get_canvas_text
Pages and canvas add_page configure_canvas
Boards list_boards inspect_board add_board
Layout templates list_layout_templates get_layout_template load_layout_template set_layout_slot_role
Canvas elements add_element add_table add_math remove_element
AI layouts and diagrams generate_layout generate_diagram generate_mind_map
Layout analysis analyze_current_layout suggest_improvements
External images search_internet_images
Agent safety set_ui_lock

How we built it

We built a React/Vite canvas with shared WebMCP tools, board-aware placement, revision checks, UI locking, AI generation, and reusable layout templates.

Challenges we ran into

We had to handle stale tool states, precise canvas coordinates, secure API keys, responsive controls, and keeping agent actions synchronized with the visible UI.

Accomplishments that we're proud of

The agent can safely inspect and modify a live canvas while users retain control and can edit every generated element.

What we learned

We learned that reliable agent experiences need clear tool contracts, strict validation, shared state, human recovery controls, and careful visual verification.

other project of webmcp: agent can draw in tldraw canvas : https://devpost.com/software/minecraft-builder agents can play music : https://devpost.com/software/harmonium-music-intrumental-webmcp agent generate pixel art : https://devpost.com/software/pixel-generator-ibjpfc

Built With

  • framer-motion
  • geogebra
  • google-gemini-api
  • html2canvas
  • jspdf
  • katex
  • lucide-react
  • mermaid
  • p5.js
  • pdf.js
  • pptxgenjs
  • react
  • typescript
  • unsplash-api
  • vercel-serverless-functions
  • vite
  • webmcp
Share this project:

Updates