Counterfactual

Dynamic paths
Home page
Flight Recorder for tool calls
Decisions, assumptions, and goals
Testing with WebMCP tools Chrome extension
Tutorial mode

Inspiration

AI agents are becoming increasingly capable of taking actions across the web, but users do not always get a clear picture of the consequences before those actions happen. A decision like buying a car can affect several competing goals at once, yet most financial tools reduce that complexity to a single static calculation.

I built Counterfactual around a simple question: what if an agent could show you several possible futures before asking you to commit to one?

Instead of immediately executing an action, Counterfactual gives the agent and user a shared space to explore alternatives, revise assumptions, understand tradeoffs, and decide together which future—if any—should become real.

What it does

Counterfactual is a WebMCP-powered decision simulator and permission layer for the agentic web.

Through WebMCP, an AI agent can discover structured tools exposed by Counterfactual, read the user's financial assumptions, create alternative scenarios, simulate their outcomes, and compare them. Counterfactual turns those scenarios into explorable futures with dynamic visualizations that show how each decision affects the user's finances and protected goals.

Users can change an assumption at any time and immediately see every future recalculate.

A built-in WebMCP Flight Recorder makes the agent's behavior visible by recording each tool it discovers and invokes, along with its arguments, risk level, origin, status, and results. Rather than receiving only the agent's final recommendation, users can inspect the actual sequence of actions that produced it.

Most importantly, Counterfactual distinguishes between exploring a future and making it real. Read-only and simulation tools can run freely, but when an agent attempts a consequential action, Counterfactual pauses execution and requires explicit human approval.

The result is a simple principle: agents can explore autonomously, but humans retain control over commitment.

How I built it

I built Counterfactual with Next.js, TypeScript, Tailwind CSS, and client-side state. Its scenario engine uses deterministic calculations so the agent's conclusions and the visual interface always operate from the same source of truth.

Counterfactual exposes structured WebMCP tools through document.modelContext.registerTool(). These include tools for reading financial state, defining a decision, protecting goals, creating scenarios, comparing outcomes, simulating purchases, and committing to a selected scenario.

Each tool has a natural-language description, JSON Schema input definition, validation, and metadata describing its behavior and risk.

To support browsers that do not yet implement WebMCP natively, Counterfactual uses the @mcp-b/webmcp-polyfill package. A thin compatibility layer detects the available context—native, polyfilled, legacy, or fallback—and registers the same tool surface against it.

Every invocation, whether initiated manually or by an external agent, passes through the same execution and logging pipeline. Calls are timestamped and risk-tagged before appearing in the Flight Recorder.

Consequential actions go through an additional policy and approval layer before they can modify committed state. The approval is tied to that specific proposed action, preventing an agent from silently turning a simulation into a real commitment.

I also tested the WebMCP integration end-to-end using Google's webmcp-evals CLI. That testing uncovered an issue that was invisible through Counterfactual's own UI but appeared when a tool was invoked through the actual WebMCP path. Fixing it helped ensure the project works through the protocol itself rather than only through a scripted demo.

Challenges I ran into

The biggest challenge was building against an experimental standard whose API and browser support are still evolving. Counterfactual needed to target the current document.modelContext API while remaining usable in browsers and agents relying on earlier or unsupported implementations.

Another challenge was maintaining a single source of truth across the agent, editable controls, scenario calculations, visual branches, and Flight Recorder. A tool call could not simply generate an answer—it had to operate on the same live state the user was seeing and immediately update the rest of the experience.

The final challenge was designing meaningful human oversight without making the agent feel unnecessarily constrained. Requiring approval for every tool call would defeat much of the value of an agent, while allowing every action to execute automatically would remove meaningful user control.

Counterfactual therefore treats risk as a gradient: agents can read, simulate, fork, and compare independently, while actions that create real commitments stop at the point where human judgment matters most.

Accomplishments that I'm proud of

I'm proud that Counterfactual uses WebMCP as a core interaction model, rather than adding it as a superficial integration. The agent does not simply fill out a form or operate a scripted demo—it collaborates with the user inside a shared decision environment.

I also built a complete flow from tool discovery → simulation → comparison → recommendation → approval.

Users can see what the agent is doing, inspect how it reached its recommendation, modify the assumptions themselves, explore alternative futures, and ultimately reject or approve the proposed action.

Most importantly, Counterfactual turns an abstract AI safety problem into something tangible: consequential agent actions become forkable futures that people can inspect before making them real.

What I learned

I learned that WebMCP is more than a reliable way for agents to interact with websites. It can enable entirely new forms of human-agent collaboration.

Structured tools allow an agent and an interface to share the same capabilities and application state without relying on fragile DOM interpretation or browser automation.

I also learned that confirmation alone is not enough to provide meaningful user control. Before approving an action, people need to understand what they are approving, which goals may be affected, what alternatives exist, and whether the action can be reversed.

Building Counterfactual showed me how structured tool schemas, deterministic application logic, risk classification, and visual design can work together to make agent behavior more transparent, understandable, and trustworthy.

Why my use case is a strong fit for WebMCP

Many agent experiences focus on reading the web: searching, summarizing, and answering questions. Counterfactual focuses on the harder problem—what happens when an agent can act on something that matters?

That is where WebMCP becomes especially valuable.

Counterfactual's decision process maps naturally onto a set of well-defined tools such as get_financial_state, define_decision, set_protected_goal, fork_scenario, compare_scenarios, and commit_scenario.

Those tools also span a genuine risk gradient. Reading financial state is fundamentally different from simulating a hypothetical decision, and both are fundamentally different from committing to one.

Counterfactual uses that distinction to determine when an agent can proceed independently and when control must return to the user.

Instead of bolting permissions onto a chat interface after the fact, the application's capabilities, schemas, risk levels, and guardrails are part of the same structured tool environment the agent discovers and uses.

How it creates a better user experience

Instead of interacting with a black box that eventually reports, "I looked into it and think you should wait," Counterfactual lets the user see what the agent actually explored.

Each hypothetical future becomes a visible branch with its own projected balance, goal impact, and explanation. The Flight Recorder shows the sequence of tool calls that produced those futures, including their arguments, risk levels, and results.

That makes questions like "Why did the agent recommend this?" answerable by inspecting what actually happened rather than trusting a secondhand explanation.

Counterfactual also includes a Manual Tool Console that lets users invoke the same tools themselves through the same schemas and Flight Recorder pipeline. The human and the agent are therefore operating the same system rather than two separate interfaces.

And when an action would turn a hypothetical future into a real commitment, execution stops. Counterfactual shows the consequences and requires explicit approval before anything changes.

The user sees the future before they inherit its consequences.

What people and agents can do together that was difficult before

Agent interaction with websites is often an all-or-nothing choice: either the agent can only read and advise, or it receives broad browser automation capabilities and starts clicking through an interface on the user's behalf.

WebMCP lets Counterfactual create something in between.

An agent can discover the page's capabilities, read financial state, create as many hypothetical futures as needed, and compare them autonomously because none of those actions creates a real-world commitment.

But when it reaches commit_scenario, control changes hands.

The same protocol therefore supports both autonomy and oversight. Whether the agent proceeds independently or requires human participation depends on the consequence of the specific tool call—not simply whether the agent has access to the website.

Because the tools are exposed through document.modelContext, the experience is also not limited to Counterfactual's own scripted agent. External agents that can inspect the live page can discover the same tools, operate the same interface, and encounter the same guardrails.

That creates a different model of human-agent collaboration: let the agent explore the decision space, then bring the human back in precisely when exploration becomes commitment.

How I implemented WebMCP

Counterfactual installs the @mcp-b/webmcp-polyfill package when WebMCP is not available natively, allowing its tools to exist on the actual document.modelContext surface rather than only inside the React application.

A thin WebMCP client detects the available context and registers each tool through the target API using structured JSON Schema inputs, tool annotations, and AbortSignal-based cleanup.

The registered tools cover the entire decision workflow, from reading state and defining goals to creating scenarios, comparing outcomes, and committing to a decision.

Each tool is assigned a risk level that powers both the Flight Recorder and Counterfactual's policy layer. Low-risk exploration can proceed automatically, while the external-commitment tool is gated behind explicit human approval before its effects can be applied to committed state.

Every invocation—whether it comes from the Manual Tool Console or an external agent—is recorded through the same pipeline.

Finally, I validate the integration with webmcp-evals, which invokes the tools through the actual document.modelContext path end-to-end. This helped catch integration bugs that would have remained invisible if I had tested only through Counterfactual's own interface.

What's next for Counterfactual

The next step is to expand Counterfactual from a single financial decision into a general preview layer for consequential WebMCP actions.

Future versions could evaluate travel bookings, subscription changes, job offers, and other decisions where an agent's action can affect multiple goals at once. This could be done with the help of multiple MCP servers.

An enhancement to make this more product-ready would be to link the user's current financial status to their bank account balance, instead of having the user prompt the agent to fill it or manually enter these details.

Additional improvements could include uncertainty ranges, source-backed assumptions, reversible-action support, reusable policy rules, and personalized definitions of acceptable risk.

The long-term vision is to give the agentic web a universal "preview before you commit" layer—one where agents can act efficiently without taking meaningful control away from people.