What it does
Codex uses WebMCP to draw, arrange layouts, explain maths, create diagrams, find images, build tables, and run interactive presentations.
Tool vocabulary
| Domain | Tools |
|---|---|
| Discovery | list_tools describe_tools |
| Canvas reading | capture_canvas get_canvas_text |
| Pages and canvas | add_page configure_canvas |
| Boards | list_boards inspect_board add_board |
| Layout templates | list_layout_templates get_layout_template load_layout_template set_layout_slot_role |
| Canvas elements | add_element add_table add_math remove_element |
| AI layouts and diagrams | generate_layout generate_diagram generate_mind_map |
| Layout analysis | analyze_current_layout suggest_improvements |
| External images | search_internet_images |
| Agent safety | set_ui_lock |
How we built it
We built a React/Vite canvas with shared WebMCP tools, board-aware placement, revision checks, UI locking, AI generation, and reusable layout templates.
Challenges we ran into
We had to handle stale tool states, precise canvas coordinates, secure API keys, responsive controls, and keeping agent actions synchronized with the visible UI.
Accomplishments that we're proud of
The agent can safely inspect and modify a live canvas while users retain control and can edit every generated element.
What we learned
We learned that reliable agent experiences need clear tool contracts, strict validation, shared state, human recovery controls, and careful visual verification.
other project of webmcp: agent can draw in tldraw canvas : https://devpost.com/software/minecraft-builder agents can play music : https://devpost.com/software/harmonium-music-intrumental-webmcp agent generate pixel art : https://devpost.com/software/pixel-generator-ibjpfc
Built With
- framer-motion
- geogebra
- google-gemini-api
- html2canvas
- jspdf
- katex
- lucide-react
- mermaid
- p5.js
- pdf.js
- pptxgenjs
- react
- typescript
- unsplash-api
- vercel-serverless-functions
- vite
- webmcp
Log in or sign up for Devpost to join the conversation.