Inspiration
I was reading through Chrome's WebMCP security guidance and stopped on one line: "it's impossible to guarantee safety inside of a large language model."
That's a remarkable thing to put in your own documentation. And it sat oddly against every WebMCP demo we'd seen, which were all sunny happy-path stuff — book a flight, add to cart, fill a form. Nobody was demoing the other side.
The thing that bothered us is structural. A WebMCP tool hands an agent real power inside a logged-in session, and the values that come back from those tools land directly in the model's context. So the moment one of your tools returns content that somebody else wrote — a support ticket, a review, an uploaded file — you've handed a stranger a writing surface inside your agent's instructions.
There's plenty of written advice about this. There was nothing you could actually run. I wanted a thing you could put in front of a developer and say "watch this," and then "now watch it not work."
What it does
I was reading through Chrome's WebMCP security guidance and stopped on one line: "it's impossible to guarantee safety inside of a large language model."
That's a remarkable thing to put in your own documentation. And it sat oddly against every WebMCP demo we'd seen, which were all sunny happy-path stuff — book a flight, add to cart, fill a form. Nobody was demoing the other side.
The thing that bothered us is structural. A WebMCP tool hands an agent real power inside a logged-in session, and the values that come back from those tools land directly in the model's context. So the moment one of your tools returns content that somebody else wrote — a support ticket, a review, an uploaded file — you've handed a stranger a writing surface inside your agent's instructions.
There's plenty of written advice about this. There was nothing you could actually run. We wanted a thing you could put in front of a developer and say "watch this," and then "now watch it not work."
How I built it
The app itself is deliberately unremarkable: React front end, a small Express backend, mock data, nothing real touched or sent anywhere.
The WebMCP layer uses document.modelContext.registerTool(), running behind chrome://flags/#enable-webmcp-testing on localhost. A hackathon is about the only place where "Chromium-only, behind a flag" costs you nothing, which is partly why I picked this project.
The decision that shaped everything else was building our own agent instead of leaning on a browser-built-in one. Our assistant panel posts to a single backend endpoint that runs an LLM with function calling over the same tool schemas. It's more work, but I needed two things a third-party agent can't give you: the reasoning trace on screen, and a demo that behaves the same way twice.
The mitigation toggles were more interesting to build than I expected. Since several of them change the shape of the tool set rather than just its contents — two-step confirmation splits one destructive tool into a preview_ / confirm_ pair — flipping a switch tears down the whole registration and rebuilds it. Each config owns an AbortController, I abort it, and re-register from a different wrapper. Watching the tool list rewrite itself in the Model Context Tool Inspector side panel turned out to be a nice bonus visual I hadn't planned on.
The payload library is just JSON with metadata, kept separate from the app on purpose so it can outlive it.
Challenges I ran into
The API had moved and half the internet hadn't noticed. I lost most of the first evening to tutorials written against navigator.modelContext and hyphenated HTML attributes like tool-name. Both are outdated — it's document.modelContext and toolname now — but the old form still works on some builds, which is the worst possible failure mode, because you get partial success and assume you're fine. I ended up going straight to the spec and Chrome's docs and treating every blog post as suspect.
The two primary sources disagree about executeTool. The spec's IDL says it takes an object. Chrome's documentation and every Chrome example pass a JSON string. I burned real time assuming I would made a mistake before working out that no, the sources just don't match yet. I feature-tested and moved on.
Tools silently not existing. For an hour our tools registered fine on one branch and were absent on another, with no error. Turns out registration rejects with a SecurityError if the document's agent cluster isn't origin-keyed, and I weren't catching the promise. It looked like the API was missing. Wrapping registration in a try/catch and actually logging it should have been the first thing I did.
Getting the attack to land was harder than expected, which is a strange thing to be frustrated by. Modern models refuse the crude payloads a decent fraction of the time, so our first demo run was flaky in the least convincing direction. I built the tiered payload library partly to solve this, and it forced a genuinely useful conversation: model-level refusal isn't a defense you can rely on, because it's probabilistic and it changes with every model release. That went from being our demo bug to being one of our talking points.
There's no confirmation primitive. Neither the spec nor Chrome's API gives you a way to say "make the human approve this." I had to design the two-step pattern ourselves, and the property that actually matters took us a while to articulate: the destructive step has to depend on a human interaction the model cannot perform. A dialog the agent can dismiss isn't a gate.
The output budget fought the demo. Chrome recommends keeping tool output under about 1.5K characters, and full ticket bodies with payloads in them blow straight past that. Truncating them weakened the attack; not truncating them was bad practice I were supposed to be teaching against. I compromised by returning summaries with a follow-up detail tool, which is what you'd do in a real app anyway.
Accomplishments that we're proud of
The toggle-and-rerun loop works, and it's visceral in a way I didn't fully anticipate when I sketched it. Explaining prompt injection takes five minutes and people nod politely. Showing delete_account fire on the words "summarize my open tickets" takes ten seconds and people sit up.
The resilience score turned the whole thing from a demo into something you can poke at. Once there's a number, people immediately start asking "what if I only turn on these two?" — which is exactly the question I wanted them asking.
I am proud that the payload library stands on its own. It's not welded to our app, so it's useful to anyone testing any WebMCP tool set, and it's the piece most likely to still matter in six months.
And showing the reasoning trace was the right call. Without it, the attack is a magic trick. With it, it's a lesson.
What we learned
The defenses that actually hold are structural, not textual. Delimiting untrusted content and labeling it helps — measurably, in our scoring — but it's mitigation, not prevention. The only thing in our matrix that held against every payload was the two-step confirm backed by a real human click, plus server-side authorization that doesn't care what the agent decided. Everything else raises the cost of the attack rather than eliminating it.
untrustedContentHint is advisory. I went in half-expecting it to be a hard boundary. It isn't. It's a signal to the agent, and what the agent does with it is the agent's business. That reframed how i think about the whole annotation system — these are hints for cooperative agents, not enforcement.
Server-side authorization is the actual backstop. By the end I would stopped thinking of the tool layer as a security boundary at all. A tool call is just an untrusted client request that happens to arrive holding the user's cookie. If your backend would refuse it from a hostile client, you're fine. If it wouldn't, no amount of clever tool design saves you.
Tool descriptions are security-relevant code. Two of our payloads only work because two tools have descriptions similar enough for the model to confuse them. That's a defect in prose, not in logic, and it should go through review like anything else.
What's next for ToolTrap - a WebMCP prompt-injection playground
Splitting the payload library into its own repo with the mitigation matrix as documentation, so it's usable without our demo app attached.
Bring-your-own-tools mode is the feature I most want and didn't have time for: paste in your real registerTool() definitions, run the payload suite against them, get a report. That's the version that stops being a teaching toy and starts being something you'd run before shipping.
I would also like to build the cross-tab variant — a hostile tab open alongside a banking tab — because the spec's security section openly acknowledges that cross-origin and cross-tab isolation is unfinished business, and there's no demo of that gap anywhere. It's the scariest scenario and the least illustrated.
Longer term: wire the payload suite into CI so a change to a tool description that makes it more confusable fails a build.
And I want to write up what I found and take it upstream to the WebMCP repo. A few of the things that cost us hours — the executeTool input disagreement, the silent-looking SecurityError — are documentation problems more than API problems, and they're fixable.
Log in or sign up for Devpost to join the conversation.