Interlock

An approval layer between AI agents and PayPal: the agent proposes a refund, an invoice or a dispute concession, and the server decides whether it runs, waits for a human, or is refused.

Comment

For judges, in 30 seconds

  • Who it's for: merchants, and the developers who build their support agents, who want an AI to handle refunds, invoices and disputes without handing it the keys to the merchant's PayPal money.
  • The problem: PayPal's Agent Toolkit gives an agent tools that move money: refunds, subscription cancellations, dispute concessions (paying the buyer's claim) and invoices. The merchant keeps the liability (PayPal Developer Agreement §1.3), and the toolkit has no per-action approval step: it can switch whole product areas on or off, but not ask before a refund. Agent apps can ask "allow this tool call?", but the person sees only what the agent says it is doing, not the real request next to PayPal's own record.
  • What Interlock does: the agent never holds PayPal credentials; Interlock's server does. The agent can only file a proposal. Interlock compares what the agent claimed, what the request would actually do, and what PayPal has on record, then runs it, queues it for a human, or refuses it. The agent's claim is shown but never used to decide.
  • The AI: a live gpt-4.1 agent works a support inbox, and gpt-4.1 scores each proposal it files next to a deterministic rule score. The model can raise a score, never lower it, and never approve anything: a score of 70 or more never runs unattended, whatever the rules say. The starting demo queue is scored by rules alone to cap model spend on a public URL, so run the agent to see both scores.
  • Try it: open the demo, press "Use the demo reviewer token", and follow the three-step Start here card.

Inspiration

Agentic commerce is arriving through the merchant's back office: refunds, disputes, invoices and subscriptions, where money leaves the merchant's own account. Buyer-side approval already exists, because a payer approves a payment before it is captured. On the merchant side, the toolkit has no checkpoint of its own. Once an agent holds the merchant's API credentials it can move money unasked, and the merchant answers for whatever it does.

It is for a merchant who lets an AI agent work the support inbox and back office: the agent does the reading and drafting, and a person on the team says yes before money leaves.

Two concrete ways an agent gets money out without anyone noticing:

  1. Understate the amount. It says $12 and puts 9,500.00 in the request.
  2. Omit the amount. A capture is the money PayPal actually collected for one payment, and refunds come out of it. When a refund request carries no amount, PayPal refunds the entire capture. A policy that reads "no amount" as "small" hands over everything.

An agent that reads customer email can also be steered by a stranger's text. We wanted the safety to hold even when the agent is fooled.

What it does

  • Real execution, only after a yes. When the reviewer approves, the server makes the PayPal sandbox call and records the id PayPal returns (in the video, refund 2E591432P2345945V). A held proposal never reaches PayPal, and a test asserts it.
  • Three versions of every action, side by side. Each proposal shows what the agent said ("refund $12"), what the request will actually do (refund 9,500.00), and what PayPal's record shows (the captured amount, whether part of it is already refunded, and what the merchant kept after PayPal's fee). Policy decides on the last two only.
  • A policy engine with editable rules, evaluated top to bottom. Anything no rule matches waits for a human, so there is never a fall-through to execution.
  • Guards outside the rule set. An amount that can't be established, a contradiction from PayPal, a lookup that fails, or a risk score of 70 or more never runs unattended, even when someone switches a rule to auto-approve. These checks are in code; no rule, reviewer or agent can turn them off.
  • Two-layer risk scoring, shown separately. Deterministic rules set a floor, and gpt-4.1 reads the specific request and may raise it. The reviewer sees both numbers and the model's reason.
  • Two roles. The agent's token can read and propose; it gets a 403 on approve, reject or rule changes. A compromised agent can't approve its own work.
  • A live agent in the console. It works a three-message inbox with paypal_read and propose_action, and the run streams step by step. One message carries a prompt injection.
  • A hash-chained, tamper-evident audit log. The demo has a button that edits a row in the database so you can watch verification catch it, and a button that undoes the edit.

How we built it

  • PayPal Agent Toolkit (@paypal/agent-toolkit 1.11), used in both directions:
    • Interlock gates 9 of its write tools: refunds, invoice sends and cancellations, subscription changes, dispute concessions, draft invoices and orders.
    • Its read tools (get_invoice, get_dispute, show_subscription_details, …) are how Interlock checks a proposal against PayPal's own record before deciding.
    • The other 23 of the toolkit's 47 tools, pay_order among them, are unreachable through any Interlock route. Interlock is default-deny.
  • PayPal REST API for the one thing the toolkit has no tool for: reading a capture (GET /v2/payments/captures/{id}). Transaction search isn't real time, so a fresh capture is missing from it.
  • Schemas. Proposal parameters are validated against the toolkit's own schemas when they are filed, so a malformed request is refused before money is meant to move.
  • OpenAI gpt-4.1 over the Chat Completions HTTP API, with no SDK:
    • Strict JSON-schema output for the risk layer.
    • A function-calling loop for the agent. Its arguments are validated against each tool's Zod schema before the tool runs, and a run is capped at 10 turns.
    • The two jobs can be pointed at different models.
  • AG Grid Community (v36) is the review queue: rows ranked by risk, a custom cell showing the rules score and the model score separately, an amount cell that says where the figure came from (the request, PayPal, or the agent's differing claim), multi-row selection, and rows that update live as the agent files proposals.
  • React + Vite console, Express API, SQLite (better-sqlite3), and server-sent events so the queue moves when an agent files something.
  • Render runs the demo as one Blueprint service from render.yaml. The API serves the built console from one origin. The free plan has no disk, so the database is reseeded on each cold start, and model spend on the seed and on agent runs is capped by settings in the blueprint.
  • Tests: 38, with no network and no model, because a guarantee that needs a language model to agree with it is not a guarantee. They cover both money-out attacks, the honest path still running unattended, a held proposal never reaching PayPal, a refund larger than its capture, an already refunded capture, a failed lookup never read as a pass, a high score held even with the rules edited to auto-approve, and the audit chain catching an edited and a deleted row.

AI tools used:

  • In the product: OpenAI gpt-4.1, for the support agent and for risk scoring.
  • In development: an AI coding assistant (Claude Code) helped write and review the code, the tests and this text.
  • For the video: Cartesia text-to-speech for the voice-over.

Challenges we ran into

  • The agent is nondeterministic, so the design can't depend on it. On the same injection email, gpt-4.1 has proposed the correct $25 refund, and on another run it has proposed refunding the whole $4,200 capture. Interlock queues both for a human. The capture is partly refunded and PayPal doesn't report what is left, so the amount can't be confirmed. We stopped trying to make the agent clever and made it not matter.
  • What PayPal does and doesn't tell you.
    • A partly refunded capture reports its state but not the remaining balance, so Interlock knows the ceiling but not the headroom, and holds the proposal.
    • Two of the sixteen allow-listed reads can't answer in the sandbox: get_merchant_insights is refused outside live mode, and get_shipment_tracking needs a scope the app doesn't hold. They return 501 unavailable_here with the reason instead of a bare gateway error.
  • The toolkit never throws. A failed PayPal call comes back as an ordinary result, either {ok:false, status} or {error:{…}}. An early build recorded a refund that PayPal rejected with a 404 as executed. Interlock now checks both shapes and records the action as failed, and a test pins each shape.
  • A public demo with published tokens. Agent runs are capped per instance, a high score never runs unattended even if someone edits the rules, and the tamper button can be undone, so one visitor can't spend the model budget, empty a shared capture, or leave a broken chain for the next.

Accomplishments that we're proud of

  • A rule that holds up: policy never decides on what the agent says about itself. Both money-out attacks are caught by facts, not by asking the agent to behave.
  • Three versions of one action on one screen, including PayPal's own record: it is what exposes the understated refund, the refund with no amount, and the refund larger than its capture in the demo queue.
  • Honest labels throughout. "Rules" and "model" are separate numbers, an unknown amount is shown as unknown, and the README has a Known limits section with a test named KNOWN LIMIT that asserts a weakness.

What we learned

  • Treat the agent's rationale as untrusted input, the same as the customer email it read.
  • A guard that must not be switched off doesn't belong in an editable rule set.
  • Read PayPal's response codes and states closely: a refund with no amount, PARTIALLY_REFUNDED, and the gap in transaction search each shaped a rule.

What's next for Interlock

  • Anchor the audit chain's tip outside the database, so truncating the tail is detectable too.
  • Per-person reviewer identity and a second approver above a threshold.
  • Run as an MCP server, so any agent host can put Interlock in front of the toolkit without code changes.

Built With

Share this project:

Updates

Submission history