Ad Pilot

Hundreds of ads, one hierarchy, zero manual drill-down — an AI agent operates your ads workspace through WebMCP, humans approve every spend.

Comment

Inspiration

This came from a problem I actually had. Professional ad platforms — Meta Ads, Google Ads, TikTok Ads, and every other major network — are complex enough that running them well basically requires a dedicated role: an ads specialist whose whole job is watching the account. I've had to be that role myself, drilling down campaign → ad set → ad, one level at a time, to find the one underperforming piece before it burns budget.

That hierarchy isn't small. I've run a single campaign with hundreds of creatives inside it on platforms built exactly this way. Manually tracing performance through that many nodes doesn't scale to one person, no matter how good the dashboard looks or which network it's attached to.

That's the gap WebMCP let me close rather than automate around. Most "AI for ads" tools are chat wrappers that talk about your campaigns while you still click through every level yourself. WebMCP made it possible for an agent to walk that same campaign → ad set → ad tree through structured tools and surface the one thing that matters, instead of me paging through tables.

But an agent that can traverse and act on hundreds of ads unsupervised is its own risk, because budget changes are real money. So the other half of the project became just as important: proving the agent can do the operational digging without ever getting to unilaterally spend.

AdPilot targets a provider-neutral ad account model rather than any one network by name (see the AdsProvider abstraction below), precisely because the underlying pain — a hierarchy too deep for one person to supervise by hand — isn't specific to any single platform.

What it does

AdPilot is an agent-native advertising workspace. A marketer sets a goal (for example: 100 purchases on a Rp15,000,000 budget), and an AI agent can:

  • read account, campaign, ad set, and ad performance through 14 read tools
  • detect anomalies and generate evidence-backed optimization recommendations
  • build campaign structure (create_campaign → create_ad_set → create_ad), always created paused so nothing can spend without a human turning it on
  • request budget or delivery changes through three gated write tools, which are recorded as approval requests rather than applied

Twenty tools in total: fourteen read, six write, three of those gated.

Authorization is actor-aware, not tool-aware. This is the part I'd point at first. When the agent calls update_ad_set_budget, the request is held in a review queue and the account does not change. When a person clicks the same control in the dashboard, it applies immediately and is logged as already decided. The queue exists to separate whoever proposes a change from whoever decides on it; when the requester is already the approver, that separation doesn't exist, so routing them to approve their own click would be ceremony rather than oversight.

The one thing the agent cannot do is clear its own request. Approval is intentionally not a WebMCP tool — it's a server action reachable only by a human clicking in the review queue. Ask the agent to approve, and all it can report is that no such tool exists.

How we built it

  • Next.js 16 (App Router, Turbopack) + React 19 + TypeScript, Tailwind for styling, Recharts for charts.
  • WebMCP tool registry in WebMcpRuntime.tsx, registering every tool via document.modelContext.registerTool (falling back to navigator.modelContext, since the surface is still moving), unregistered cleanly with an AbortSignal on unmount.
  • Contracts as the single source of truth: tool name, description, and JSON Schema live in lib/webmcp/contracts/, consumed by both the browser registration code and the server-side dispatch route, so the schema a judge reads is the schema that actually runs.
  • Thin handlers over shared services: every tool handler is an adapter over the same application services the dashboard calls. There is no duplicated business logic between "the UI path" and "the agent path" — the dashboard's own create forms and pause controls go through the same handlers, and the only thing that differs is the authority of the caller.
  • AdsProvider abstraction: DemoAdsProvider is a deterministic, in-process implementation seeded from a fixed dataset, so every run produces identical numbers for judging. A network-backed provider can be added behind the same interface without touching the tool contracts.
  • Approval pipeline: gated writes (update_ad_set_budget, update_entity_status, apply_recommendation) land in a pending-change queue when an agent asks. Approving or rejecting is a human-only server action, gated by session, never exposed as a callable tool.
  • Annotations that mean something: readOnlyHint follows the contract kind, and untrustedContentHint is set for every tool whose payload can carry text a person or an earlier agent wrote — entity names, creative copy, approval reasons. Only four tools return purely app-owned configuration or computed numbers, and those are a documented opt-out list, so a tool added later is treated as untrusted by default.

Recommendation logic is deliberately simple and explainable rather than a black box. An ad set is flagged as an anomaly when its cost per acquisition exceeds the median across active ad sets by a fixed multiple:

$$ \text{flag if } \mathrm{CPA}{\text{ad set}} > 1.5 \times \mathrm{median}\left(\mathrm{CPA}{\text{active ad sets}}\right) $$

A reallocation is only proposed when the spread between the worst and best performing ad set is wide enough to be worth acting on:

$$ \mathrm{CPA}{\text{worst}} > 1.4 \times \mathrm{CPA}{\text{best}} $$

An ad is flagged for creative fatigue when its click-through rate drops more than 10% against the previous week. ROAS is reported as a plain ratio,

$$ \mathrm{ROAS} = \frac{\text{Revenue}}{\text{Spend}} $$

with an explicit null (rendered as "—") instead of a fabricated 0 when spend is zero, because a dashboard that prints fake numbers is worse than one that admits it doesn't have data yet.

Challenges we ran into

The hardest problem wasn't WebMCP itself. It was making sure the product's central claim — a human approves every high-impact change — was true at the infrastructure level, not just in the happy-path UI flow.

During a self-run adversarial review I found the approval endpoint could be reached directly, with no session, by replaying the review page's own form fields. The write tools were correctly gated behind human review, but review itself had no authentication in front of it: anyone who found the URL could approve or reject a pending budget change. That's a sharp bug in a project whose entire pitch is "humans keep control," so I treated it as blocking. I added a signed, server-revocable session (Web Crypto HMAC over a random session id, checked against a server-side active-session set so logout can't be bypassed by replaying an old signed cookie), gated every route except /login in proxy.ts, and re-verified with raw HTTP requests that an unauthenticated caller now gets a 401 or a redirect instead of a state change.

A second audit pass found a quieter but nastier bug. Number("") and Number(" ") both evaluate to 0, and update_ad_set_budget accepted a floor of zero — so an empty form field, or a malformed agent argument, produced an approval request that read as a perfectly legitimate decision while actually setting an ad set to Rp0 per day. I confirmed it against a running server before fixing it: "", " " and 0 all returned HTTP 200 and queued a change, and only a negative value was refused. All four now return 400. Stopping delivery is update_entity_status, not a budget of zero.

The same pass caught three entries in the seeded activity log that named tools this app has never exposed (get_ad_performance, build_campaign_plan, get_account_overview). Agent Activity is the surface where someone checks that tool calls really happened, so phantom rows there undermine the exact thing it exists to prove. A fourth seeded row claimed a change was "queued for human approval" while the queue was empty on a fresh boot. All four are fixed, and I now cross-check every logged tool name against the registry.

I also measured the tool surface against Chrome's published WebMCP tool guidance instead of assuming it complied. One tool name was 32 characters against a recommended 30. One description sat at exactly 500 of 500, with no headroom. Worst of all, list_tool_executions could return the audit log 100 entries at a time — about 22,000 characters in one tool output, against a recommended budget of roughly 1,500. It's capped at 25 now and defaults to 10. Six read tools still exceed 1.5K and I documented that as a deliberate trade-off rather than pretending otherwise: the evidence an agent needs to justify a budget change doesn't compress that far without dropping the numbers the reasoning rests on.

The other recurring challenge was that WebMCP support is genuinely uneven across clients right now. Tool discovery worked cleanly in more than one surface, but invocation behaviour differed — one client reported its WebMCP connection as unavailable in a particular mode even though the same server-side handlers ran fine when called directly. I leaned on an HTTP-level test path (calling /api/webmcp/<tool> directly with a session cookie) as the ground truth for "does the handler actually work," independent of which agent client happened to be cooperating that day.

Smaller but real: a metric drifted because the same recommendation could be queued twice, fixed by constraining recommendationId to a fixed enum and making apply_recommendation return the existing open request instead of creating a duplicate. And a root-level loading state was quietly turning a real 404 into a 200.

What we learned

  • An approval gate is only as strong as its least-obvious entry point. It's easy to gate the tool call and forget that the human-facing review UI needs the same rigour, especially when it's "just" a button click.
  • Asking who is calling is more useful than asking what is being called. Gating by tool name produced a worse product: it made a person approve their own click. Gating by caller kept the oversight where it means something.
  • Guidance is worth measuring against, not assuming. Every budget problem above was invisible until I counted characters and diffed the log against the registry. None of it showed up as a failing test.
  • WebMCP's imperative registration API rewards small, composable tools with tight schemas over a few do-everything tools. It made the read/write split and the annotations do real product work instead of being decorative.
  • Deterministic demo data is a feature, not a shortcut. Being honest that DemoAdsProvider is synthetic, and labelling it as such in the top bar, mattered more for trust than trying to look more real than the project is.
  • Testing across multiple WebMCP-capable clients early would have caught the client-specific invocation gap sooner than treating one client as representative of WebMCP support in general.

What's next

The next step is the one the architecture was shaped for: a real network-backed provider behind the existing AdsProvider interface, starting with the Meta Ads API and followed by Google Ads and TikTok Ads. To be clear about what exists today — none of that is implemented. The shipped build runs entirely on DemoAdsProvider, and the UI labels it as such. I'd rather state the roadmap plainly than imply a connection the code doesn't have.

The interface is the easy part. The real work is everything around it:

  • OAuth and app review. Each network gates write access behind its own review process and permission scopes. Read-only access lands first; write access is a separate approval with a longer lead time.
  • Multi-user identity. The current build has one shared login for the whole demo account. That is honest for a hackathon deployment and completely insufficient for real ad accounts. Per-user auth, per-account authorization, and an audit trail tied to a real identity all have to come before the first live token.
  • Asynchronous reporting. Real insights endpoints are report jobs, not synchronous reads: submit, poll, page through results, respect rate limits. The read tools keep their current shapes, but the provider grows a job queue and a cache behind them.
  • Write safety against a live account. Idempotency keys so a retried tool call can't double-apply a budget change, plus reconciliation for changes made outside AdPilot. The approval queue gets more valuable here, not less, since a held request is the last checkpoint before real money moves.
  • A real error taxonomy. Demo failures are simple. A live network returns policy rejections, review states, billing holds, and partial successes, and each needs to surface as something a person can act on rather than a generic failure.
  • Normalisation across networks. Currency, timezone, attribution windows, and objective naming all differ per platform. The provider layer has to normalise them, or every metric comparison silently becomes wrong.

Beyond providers, two product directions I want to test: letting an agent propose a whole multi-step plan as one reviewable diff instead of a change at a time, and giving the review queue a policy layer — spend thresholds under which an agent's change applies without a human, set by the human, and revocable.

Built With

Share this project:

Updates