ProofPilot

ProofPilot Dashboard
WebMCP
Agents

Inspiration

AI agents are getting really good at researching, analyzing information, and taking actions for us. But when the decision actually matters, there is a bigger question:

How much authority should we give the agent?

Imagine a team choosing an AI coding assistant, a vendor, or a software architecture. Getting an AI to say “I recommend option A” isn't particularly difficult anymore. The harder part is understanding why it recommends A, what evidence supports that recommendation, what evidence disagrees with it, what happens if our priorities change, and whether the agent should be allowed to change the decision at all.

That was the idea behind ProofPilot:

Decisions you can interrogate.

We wanted to build a workspace where an AI agent could do meaningful work alongside a human without quietly becoming the person making the final decision.

What we built

ProofPilot is a human-controlled decision workspace where an AI agent can investigate, analyze, challenge, and propose — while the human stays in control of consequential decisions.

The workflow is simple:

Agent investigates → analyzes → proposes → human approves → state changes → audit trail.

The human starts by defining the decision, the available options, and the criteria that matter.

The agent can then work with that live decision through WebMCP. It can inspect the current decision state, search and add evidence, compare options, find contradictions, run sensitivity analysis, evaluate scenarios, and analyze risk.

But there is an important boundary.

If the agent wants to make a consequential change — for example, changing a criterion's weight or proposing the final decision — it cannot apply that change directly.

Instead, ProofPilot turns it into a pending proposal.

The human can inspect the proposal, approve it or reject it, and only after approval does the decision state change. The change is then recorded in the audit trail.

This means the agent and the human are working on the same decision, rather than the human having to copy information between a chatbot and a separate decision-making tool.

Why WebMCP

WebMCP is a core part of ProofPilot because we wanted the agent to interact with the application's actual capabilities and live state, rather than simply reading a webpage or relying on browser automation.

Instead of building a separate integration for one particular AI client, ProofPilot exposes its decision capabilities as WebMCP tools directly from the application.

We currently expose 13 tools:

get_decision_state get_evidence search_evidence add_evidence compare_options detect_contradictions run_sensitivity_analysis analyze_decision_risk evaluate_scenario propose_weight_change propose_decision request_human_review generate_decision_brief

These tools cover investigation, evidence, analysis, governance, and output.

Because they operate on the application's live decision state, a WebMCP-capable agent can investigate and reason about the same information that the human is looking at.

The key idea is simple:

Giving an agent access to a tool does not mean giving it authority to make the decision.

Read and analysis tools can execute normally. Actions that can change consequential decision state become proposals and require explicit human approval.

That's the boundary we wanted ProofPilot to enforce.

How we built it

The frontend is built with React, Vite, Tailwind CSS, Wouter, and TanStack Query. The backend uses Express, TypeScript, Zod, and Vitest.

One thing we were careful about was not making the LLM the source of truth for the actual decision score.

ProofPilot uses a deterministic multi-criteria decision engine for weighted scoring. The system also handles confidence, risk analysis, contradiction detection, evidence gaps, sensitivity analysis, and scenario evaluation.

The agent is responsible for investigating and proposing. The application is responsible for calculating and enforcing the decision logic.

WebMCP tools are registered in the frontend through the model context API. Each tool has its own description, input schema, and execution logic connected to the application's functionality.

We also built a Developer view where the registered WebMCP tools and their schemas can be inspected. We wanted the agent-facing interface to be visible and understandable rather than something hidden behind the UI.

Challenges

The hardest part wasn't registering the WebMCP tools.

It was deciding where the agent's authority should stop.

We wanted the agent to be capable enough to do real work. A read-only chatbot wouldn't be very useful.

At the same time, we didn't want an agent to quietly change weights or make a final decision just because it had access to the corresponding function.

That led us to the proposal-based governance model.

Consequential actions first become pending proposals. The human reviews them, approves or rejects them, and only then can the active decision state change.

We also had to make the underlying decision engine predictable and explainable. Rather than letting an LLM invent a final score, we separated the agent's investigation from the application's deterministic analysis.

We tested the WebMCP integration with ChatGPT's Site Tools environment and successfully exposed and executed the registered tools from the deployed application.

What we learned

Building ProofPilot changed the way we think about agentic applications.

We initially thought about giving agents more capabilities. But the more we built, the more we realized that capability and authority don't have to increase together.

An agent can be given powerful tools to investigate evidence, challenge assumptions, compare alternatives, and prepare a recommendation without being given the ability to make the final consequential change.

That's where WebMCP became especially interesting for us.

It gives the agent a way to interact with the application's real functionality and state, while the application itself can decide what each tool is actually allowed to do.

For us, the result is a different kind of agentic workflow:

The agent does the investigation. The system does the analysis. The agent makes the proposal. The human makes the consequential decision. And the entire process is recorded.

That's what ProofPilot is about.

Not just getting an answer from an AI, but being able to question the reasoning behind the answer, understand the evidence, challenge the assumptions, and stay in control of what happens next.

ProofPilot — Decisions you can interrogate.