Mandaneum

• 

Inspiration

Mundaneum is inspired by Paul Otlet’s original Mundaneum: an ambitious early attempt to collect, connect, and organize the world’s knowledge using millions of cross-referenced index cards.

Modern knowledge tools are powerful, but capturing ideas often requires deciding on folders, filenames, and structure too early. We wanted a space where people could first think naturally—dropping in notes, links, files, drawings, equations, and unfinished ideas—and let organization emerge later.

We also wanted AI collaboration to feel native to the interface. Instead of making an agent interpret every interaction from screenshots or manipulate the page through coordinates, Mundaneum uses WebMCP to expose the board’s actual content and permitted actions directly to the agent.

What it does

Mundaneum is an infinite visual research board designed for people and AI agents to work on together.

People can add notes, links, images, documents, media, files, widgets, freehand drawings, YouTube videos, Spotify songs, PDFs, Excel, and expandable document canvases. Document canvases combine positioned text, handwriting, shapes, and equations into a single visual document that a multimodal model can inspect as an image.

Users can select a card or Shift-click multiple cards to establish the current focus. WebMCP returns those selected items first, allowing ChatGPT to work on a specific group without scanning an entire large board. Selected drawings are provided as a cropped image, while cards retain their structured text and metadata. Users can also copy a card’s exact ID and reference it directly in a prompt.

Through WebMCP, an agent can:

• Read the board and its current selection
• View a document canvas or the entire board as a PNG
• Add provisional cards
• Connect related cards
• Group and label information
• Annotate cards visually
• Identify and merge likely duplicates
• Request grid, masonry, cluster, tree, row, or column arrangements
• Create sketches without controlling their final position

The central rule is that agents can contribute meaning, but they cannot provide coordinates. The page remains responsible for placement, collision detection, and layout.

How we built it

Mundaneum is a React and TypeScript application powered by Vite. Board state is stored locally in IndexedDB, while Cloudflare Workers and Durable Objects support sharing, capture, and synchronization.

The WebMCP layer registers imperative tools through document.modelContext. The same internal tool definitions are used by the visible application, ensuring that agent actions follow the same rules as human actions.

The board performs its own organization work in the browser:

• Transformers.js generates semantic embeddings in a Web Worker.
• Pairwise similarity creates a graph of related cards.
• Louvain community detection identifies clusters.
• D3 Force calculates spatial arrangements.
• Collision detection and empty-space placement prevent cards and widgets from overlapping.
• An RBush spatial index supports efficient selection and region queries.

Rich media uses a lightweight facade until the user activates it, preventing large boards from loading many expensive embeds simultaneously. Documents and spreadsheets are parsed through lazy-loaded libraries, while drawing and document canvases can be rasterized into PNG images for multimodal agents.

Stylus input uses Pointer Events, pressure data, pointer capture, coalesced events, palm rejection behavior, and hardware-eraser handling to improve writing on tablets.

ChatGPT, OpenAI Codex, and Anthropic Claude Code were used during development for research, implementation, debugging, testing, and documentation.

Challenges we ran into

One of the hardest problems was deciding how much information should be structured data and how much should remain visual. Text cards are easier for an agent to process structurally, but handwriting, equations, diagrams, and overlapping annotations lose meaning when converted into isolated text. Our solution was hybrid: preserve structured card data while supplying selected drawings and document canvases as images.

Selection also required careful design. On a large board, returning everything equally can waste the model’s context and make focused requests unreliable. We made human selection an explicit attention signal, returning selected cards first and combining selected drawings into a single cropped image.

Spatial layout created another challenge. Simple force-directed layouts often produced overlapping clusters, particularly in masonry, cluster, and tree arrangements. We separated semantic intent from geometry: agents request an arrangement, while the browser calculates positions, measures real card dimensions, and searches for available space.

Tablet drawing introduced a different class of input problems. Pen input, touch gestures, palm contact, scrolling, erasing, and card dragging can all compete for the same pointer events. Supporting pressure-sensitive writing without breaking ordinary mouse and touch interaction required dedicated pointer handling and repeated device-oriented testing.

Finally, converting the complete board into an agent-readable image was more complicated than returning its SVG representation. We added real PNG rasterization so multimodal models receive an actual screenshot-like image containing cards, links, drawings, and spatial relationships.

Accomplishments that we're proud of

We are proud that Mundaneum treats WebMCP as part of the product’s interaction model rather than as a thin demonstration layer.

The agent operates on meaningful concepts—cards, selections, links, groups, drawings, and arrangements—without controlling screen coordinates. This creates a clear division of responsibility: the person directs attention, the agent contributes understanding, and the application protects the visual workspace.

We are also proud of the document canvas. It allows typed text, handwriting, shapes, and equations to coexist in one movable and resizable document while remaining readable by a multimodal agent.

Other accomplishments include:

• Selection-aware agent context
• Multi-card selection through Shift-click
• Image-based transfer of selected drawings
• Whole-board PNG capture
• Copyable card IDs
• Non-overlapping card placement
• Resizable document and widget cards
• Pressure-sensitive stylus support
• Multiple browser-computed arrangements
• A shared tool registry for the interface and WebMCP
• A public, MIT-licensed implementation

What we learned

We learned that good agent interfaces are less about exposing every possible action and more about defining strong boundaries.

Preventing agents from setting coordinates initially appeared restrictive, but it simplified the tool surface and made agent contributions safer. The agent expresses intent, while deterministic browser code handles geometry.

We also learned that visual and structured context should complement each other. Screenshots are excellent for handwriting, diagrams, and spatial meaning, while structured data is better for exact text, IDs, links, and targeted actions. Combining both creates a much stronger interface than relying exclusively on either representation.

Human selection proved to be a powerful form of context engineering. A simple Shift-click can communicate scope more reliably than a long prompt explaining which objects matter.

Finally, pen input demonstrated how much interaction design depends on hardware. A drawing system that works with a mouse is not automatically usable with a stylus and touchscreen.

What's next for Mundaneum

Our immediate focus is testing and refining the complete WebMCP workflow across more browsers, board sizes, and tablet devices.

We also want to improve:

• Selection and movement of mixed groups containing cards and drawings
• Stylus responsiveness, palm rejection, and gesture handling
• Exporting and importing complete boards
• Collaborative editing and synchronization
• Accessibility and keyboard navigation
• Large-board performance
• More document-canvas editing tools
• Clearer review controls for agent contributions

The repository also contains early experiments for optional in-page AI providers and dynamically extending the board’s tools. These ideas are not yet part of the fully tested experience. Future work will evaluate them carefully, including security, permissions, failure handling, and user review, before presenting them as supported features.

Ultimately, we want Mundaneum to become a shared thinking surface where people can work visually and AI agents can contribute meaningfully—without taking control away from the person using the board.

Built With

• react
• typescript
• webmcp