An AI Agent and webMCP Walked Into a Museum...

Museum collections shouldn’t stay behind glass. Search major art museum collections, build a private gallery, and let an AI agent create curated art exhibitions for you. For the art historian inside.

Comment

Inspiration

Museum collections shouldn’t stay trapped behind four walls. I wanted to liberate art from institutional silos by aggregating masterpieces across major institutions - The Met, the Art Institute of Chicago, and the Cleveland Museum of Art - into a unified workspace. By pairing this open collection access with WebMCP, I set out to explore how browser-native AI agents can act as collaborative co-curators, helping everyday art lovers organize visual narratives. Let your agent surprise you with the patterns it finds! WebMCP lets your app be manageable by the AI so that it can rearrange your gallery, annotate it, and create an educational narrative about your selections.

What it does

The Museum Media Grouper and Exposition Builder is an Astro and Cloudinary-powered studio for searching, saving, and curating cross-museum artwork.

Universal Search & Ingestion: Allows users to perform live queries across three major museum APIs simultaneously, saving selected pieces into a personal Cloudinary gallery organized by artist. The system is designed to quickly onboard new APIs as we discover and vet them.

Agentic Exposition Builder: Integrates 8 custom WebMCP tools through document.modelContext so a browser-based AI agent can search saved assets, initialize draft exhibitions, sequence artwork, and write wall-card narrative captions using what it knows about the art and the connections it discovers while curating it.

Human-in-the-Loop Curation: Keeps the human curator in complete control. An agent can assemble and propose a story layout, but setting a project as ready_for_review and issuing the final publication requires explicit human verification.

How we built it

Framework & Storage: Built using Astro for fast server-side asset handling and dynamic rendering. Ingested museum objects are stored and transformed in Cloudinary under a structured museums/* namespace, with story drafts managed locally via a modular StoryRepository pattern (localStorage).

WebMCP Architecture: Exposed 8 structured tools via document.modelContext (search_saved_assets, create_story_draft, add_story_items, arrange_story_items, edit_item_text, preview_story, delete_story_draft, and request_story_review).

Dual-Path Execution: Designed UI controllers (storyBuilderController.ts) as thin DOM wrappers over shared, dependency-injected handlers in toolHandlers.ts. A user clicking a button and an AI calling a tool run the exact same function, preventing behavior drift.

Lots of help from Claude and Lovable for design inspiration.

Challenges we ran into

Every museum API, when you can find them (not every museum has an open one - sadly Boston's MFA does not!), has a slightly different shape. This project began initially as a museum explorer, to see how easily I could ingest art into Cloudinary and use tagging as a way to build a database of metadata attached to the art.

I also found that discovering a demoable use case for webMCP was more challenging than originally anticipated. I originally wanted to attach a webMCP routine to a curriculum I'm developing, but it helped only to make quizzes auto-completable, which is an antipattern. So I pivoted this project, which was inspired by work I did at Microsoft with the Cooper Hewitt Museum and, more recently, as a contractor with Microsoft Philanthropies working to stand up the new Roosevelt Presidential Library.

Accomplishments that I'm proud of

It was a delight to watch the AI Agent clicking around, arranging items, and filling in the field to create the exposition. It was able to detect patterns and I found myself learning about art as I worked with the tool. I didn't know Sonia Delaunay was from Ukraine, for example. It might turn out to be a useful tool for fledgling art historians.

What I learned

WebMCP Simplicity: Exposing client-side browser tools directly through document.modelContext eliminates fragile DOM scraping, allowing agents to interact with web apps cleanly and deterministically. But we'll have to see where the spec lands and how browsers adopt it.

What's next

Cloudinary Search API Migration: Upgrade verification logic from in-memory collection filtering to native Cloudinary Search expressions to scale beyond the initial asset cap.

Server-Backed Persistence: Swap the local StoryRepository interface for a multi-user database backend, allowing real-time collaborative curation across multiple devices. Individual auth and login would allow users to save their own galleries and share them via microsites. Currently the cool archives don't persist.

Built With

  • astro
  • claude
  • github
  • impeccable
  • vscode
Share this project:

Updates

Submission history