CacheLens WebMCP

CacheLens uses WebMCP to let ChatGPT inspect LLM traces, find prompt-cache waste, explain why it happens, and simulate fixes that cut API cost.

Comment

Inspiration

LLM applications repeatedly send large amounts of identical context: system prompts, tool definitions, policies, and conversation history. Prompt caching can make that repeated input dramatically cheaper—but only while the cacheable prefix remains stable.

A seemingly harmless change near the beginning of a prompt can invalidate cache reuse for everything that follows. Dynamic metadata, changing instructions, timestamps, or volatile content placed at the wrong cache boundary can quietly cause an application to pay to process thousands of largely unchanged tokens again.

The difficult part is that developers usually cannot see why this is happening.

That led to CacheLens: a developer tool for making prompt-cache behavior observable, identifying where prefix reuse breaks, explaining likely causes, estimating the financial impact, and recommending concrete fixes.

The WebMCP Challenge suggested a natural next step: instead of requiring developers to manually interpret every trace and diagnostic, let ChatGPT inspect CacheLens directly through structured capabilities exposed by the live application.

What it does

CacheLens WebMCP turns prompt-cache diagnostics into a collaborative workflow between the developer, the web application, and ChatGPT.

A developer can open a CacheLens trace and ask questions such as:

  • Why is my cache hit rate low?
  • Where is the cacheable prefix breaking?
  • Which content is changing?
  • How many stale tokens are being unnecessarily rewritten?
  • What is that costing me?
  • For an identified break, how much could be saved if the stable and volatile content were split correctly?

Through WebMCP, ChatGPT discovers three domain-specific tools directly from the page:

  • list_traces
  • explain_cache_behavior
  • estimate_split_savings

Rather than scraping the DOM or requiring the user to copy diagnostic data into chat, the agent asks CacheLens directly for the underlying analysis.

For example, one real captured trace reports a 44.3% provider-measured cache hit rate, 3 of 3 avoidable prefix breaks, and 28,175 stale tokens rewritten. CacheLens identifies the volatile cache boundary and can estimate the savings for a specific affected turn while clearly stating the assumptions and limitations of that estimate.

The human remains in the loop through the CacheLens interface while ChatGPT reasons over the same application state through WebMCP.

The workflow becomes:

observe → diagnose → explain → estimate → improve

How we built it

CacheLens captures and analyzes LLM request traces while preserving the structure needed to understand prompt-prefix reuse.

Its existing analysis engine compares successive requests, determines where stable prefixes diverge, separates stale tokens from genuinely new tokens, attributes likely causes, and estimates cache-related waste.

For WebMCP, we deliberately reused that analysis rather than creating a second implementation.

A thin serialization layer projects CacheLens analysis objects into safe application data. A small read-only HTTP server exposes the trace catalog and analysis endpoints, and a lightweight browser interface registers the three WebMCP tools through document.modelContext.

The WebMCP tools use constrained inputs: trace identifiers are resolved through an explicit server-side catalog rather than arbitrary filesystem paths, and the tools expose no shell commands, writes, credentials, or mutation operations.

The tools are intentionally higher-level than UI automation. CacheLens does not expose actions such as "click this button." It exposes meaningful developer operations: list available traces, explain cache behavior, and estimate the savings associated with splitting a specific identified cache break.

That became one of the most important design principles of the project:

WebMCP is most useful when a website exposes meaningful application capabilities, not simply another way to operate its interface.

Challenges

The first challenge was identifying cache waste accurately.

Prompt caching is prefix-sensitive, so a small mutation can force a much larger region of otherwise unchanged input to be rewritten. Detecting that two strings differ is easy. Determining which divergence caused expensive cache invalidation—and separating stale tokens from genuinely new ones—is considerably more useful.

The second challenge was presenting the result in terms developers care about. Token counts alone are abstract, so CacheLens connects structural changes to cache behavior and estimated cost while distinguishing measured provider usage from heuristic token estimates.

The WebMCP integration introduced another challenge: deciding what an AI agent should actually be allowed to do.

Exposing every internal operation would create a large and confusing tool surface. Exposing only low-level UI controls would miss the purpose of WebMCP. We settled on three small, read-only, domain-specific tools corresponding to real developer questions.

We also deliberately avoided a tempting feature: a general-purpose "simulate fix" tool. CacheLens does not currently model arbitrary prompt reorganizations or predict a new cache-hit rate. Instead, estimate_split_savings performs one narrow counterfactual supported by measured analysis: for an identified cache break, estimate the cost difference if the unchanged portion could have been read from cache rather than rewritten.

Keeping that boundary explicit made the result more defensible.

What we learned

The project reinforced an important lesson about AI infrastructure: optimization problems are often observability problems first.

Prompt caching can provide substantial savings, but developers cannot optimize behavior they cannot see.

We also learned that WebMCP changes the relationship between AI agents and web applications. Instead of treating a website as a collection of pixels, DOM elements, and buttons, an application can explicitly describe the capabilities it intends to offer an agent.

For CacheLens, ChatGPT does not need to guess what a chart means or scrape text from a dashboard. It can ask the application directly for the underlying diagnostic information and help the developer reason about it.

We validated this end to end in both Google Chrome with WebMCP enabled and ChatGPT's built-in browser. In ChatGPT, all three tools were discovered and invoked through natural-language requests while the page's WebMCP activity panel displayed the corresponding calls.

That combination—structured application capabilities, shared live application state, and human oversight—is what makes WebMCP particularly compelling to us.

What's next

CacheLens can expand beyond individual traces into continuous monitoring of production LLM applications: detecting regressions in cache efficiency, comparing deployments, identifying recurring sources of prompt instability, and surfacing optimization opportunities over time.

Its current ingest path focuses on Anthropic-format request traces; additional provider adapters are a natural next step.

More broadly, we see WebMCP as a way to turn developer tools from passive dashboards into active collaborators.

CacheLens started by answering a simple question:

Where is my LLM application wasting tokens?

With WebMCP, a developer can now ask that question directly—and let CacheLens and ChatGPT investigate the evidence together.

Built With

Share this project:

Updates

Submission history