AgentOps

AgentOps gives AI agents real WebMCP tools to act on the web and a Sentinel security layer that keeps you in control of every risky move.

Comment

Inspiration

Every "AI agent that browses the web for you" demo I'd seen was binary, either the AI does everything unsupervised unsettling the moment money or personal data is involved or the user has to approve every single click, which defeats the point of delegating anything. I wanted to build the middle ground: Which is an agent that acts freely on low-risk actions, but hits a real, structural trust boundary the moment something consequential is at stake. WebMCP made that possible in a way ad-hoc browser automation never could because a registered tool is structured data, a security layer can actually reason about what a call means before it runs, not just watch pixels get clicked.

What it does

AgentOps takes a plain-language goal "book me a flight to Venice under $600," "order pizza for a get-together" and turns it into a mission: a sequence of real WebMCP tool calls. Every one of those calls passes through Sentinel, our trust/policy layer, before it executes:

  • Low-risk actions (search, compare, read) run automatically.
  • Consequential ones (bookings, purchases above a user-set threshold, sending messages) pause and ask for a one-click approval, with a plain-English reason and a risk score.
  • A fixed set of categories (credentials, password changes, financial transfers) are never auto-approved, full stop.
  • Anything a tool returns is treated as untrusted content if it contains an embedded instruction trying to override the agent (a prompt injection), Sentinel blocks it outright. Not a setting. A boundary.

The user sets their own risk tolerance once in a Trust Policy screen; every mission afterward respects it without further supervision. Missions pause and resume in place no restarting and every decision is logged.

How we built it

The frontend is a Vite/React app with a Three.js "Agent Core" that visually reflects the agent's state (idle, thinking, executing, awaiting approval, blocked). Every mission step is registered as a real tool via document.modelContext.registerTool() / navigator.modelContext native on Chrome behind the WebMCP flag, or through the @mcp-b/webmcp-polyfill package everywhere else, so the same code path works across environments. Before executing anything, the mission engine calls getTools() a genuine discovery step, not a hardcoded list.

Sentinel's gate lives inside each tool's execute(), so the enforcement isn't UI-only it applies no matter who calls the tool, our own runner or, in principle, an external agent. Mission plans are generated at runtime by an LLM (via an OpenAI-compatible endpoint) rather than picked from a fixed script, and one mission's flight-search step calls a real (sandboxed) flight-search API through a small serverless proxy the only place any secret key lives, since real commerce/travel APIs explicitly block direct browser calls.

Challenges I ran into

A few things broke in ways that shaped the final architecture:

  • My first pick for real flight data, Amadeus's free self-service tier, was decommissioned mid-build. We moved to Duffel's test-mode API instead.
  • Duffel (like most real commercial APIs) explicitly blocks direct browser calls you cannot safely hold a real secret key client-side. That's what pushed me to add a minimal Vercel serverless function as a proxy: the smallest amount of backend that makes a genuinely real API call possible without leaking credentials.
  • The LLM provider deprecated the model I was building with mid-hackathon, silently breaking AI planning until I caught it and swapped to their current recommended model.
  • The most important bug we found: Sentinel's purchase-risk check used (amount || 0) > threshold, so a step with no amount attached like an AI-planned "order pizza" step that omitted a price silently defaulted to "safe" and auto-approved. I fixed this as defense-in-depth: a missing amount now escalates to "ask" rather than being treated as zero risk, and a second, independent check flags any step whose own description implies a real commitment (ordering, paying, booking, canceling) even if the planner mis-categorized it. Sentinel's judgment isn't purely downstream of a label the LLM chose.

Accomplishments that I'm proud of

Getting Sentinel to feel like a real trust boundary rather than a UI decoration the prompt-injection detection in particular, where content a tool returns is scanned as untrusted data before the agent can act on it, not just the request being sent. And getting at least one mission wired to genuinely real external data (not just simulated), despite hitting a dead API provider and browser CORS restrictions along the way, without ever compromising a secret key.

What I learned

WebMCP's ecosystem is still very early almost no production site implements it yet, which shaped an honest choice to have AgentOps register its own actions as tools rather than pretend arbitrary sites already support this. I also learned, the hard way, that any security layer built on top of an LLM's own categorization needs a second, independent check trusting a model to always self-report risk correctly is exactly the kind of single point of failure a "trust boundary" is supposed to prevent.

What's next for AgentOps

Wiring real external data into the Shopping and Finance missions the same way I did for Travel; giving AgentOps a second, fully independent WebMCP surface (a separate site registering its own tools) so the External Agent → Sentinel → WebMCP Tool → Website story is completely literal instead of self-hosted; and letting a real external agent not just our own UI discover and drive AgentOps' tools directly.

Built With

Share this project:

Updates

Submission history