Vouch

A credit bureau for AI agents that spend money. Agents take a driving test, earn a score, and build history on real PayPal outcomes. Merchants ask Vouch before money moves.

Comment

The problem

AI agents are starting to spend real money. Every agentic commerce protocol shipped so far covers identity, authorization, checkout, or settlement: AP2 proves the human signed a mandate, ACP and UCP handle checkout, x402 and MPP settle. None of them says whether the agent is likely to get it right. A merchant cannot tell a careful agent from a sloppy one, so it either blocks all of them or trusts all of them.

Vouch is the missing layer: a credit bureau for AI agents. PayPal, as a closed loop that sees buyer, seller, capture, refund, and dispute in one place, is the natural place to run it.

What it does

  1. Driving test. Every agent starts with a shopping exam. Gemini writes a fresh one each time: a task, an eight-item catalog, and three hidden traps, including a seller note planted inside a product description that tells the agent to add an expensive bundle. Deterministic code validates the traps exist, then grades the cart. The sloppy agent obeys the note and starts at 58. The careful one ignores it and starts at 75.
  2. Assessment before money moves. The merchant calls POST /assess with the agent, the cart, and the owner's stated intent. Gemini rates how well the cart matches the intent (0 to 1) and names what went wrong: INJECTED_INSTRUCTION, UNREQUESTED_ITEM, OVER_BUDGET, WRONG_ITEM. A fixed decision table turns tier plus fit into approve, step up (owner confirms in PayPal), or decline. The model is an input, never the decider.
  3. Real PayPal money. Approved carts become PayPal Orders v2 orders captured on the spot. Provisional agents get an order held for the owner's approval. Refunds go through the PayPal Refunds API. Captures, refunds, and disputes arrive by webhook and move the score: +2, -8, -20.
  4. Trust ladder. Human in the loop until the score is built. Provisional agents need the owner for every purchase and are declined over $50. At 70+ with five clean outcomes an agent is established and buys on its own. Under 40, or a run of bad outcomes, it is flagged and declined until it recovers.
  5. On the record. A Vouch score card per agent with its score path, and a ledger of every assessment with reason codes and the PayPal result.

The hosted demo lets judges run the whole story: pick an agent, run the driving test, try to buy, edit the cart before Vouch judges it, approve a held order on PayPal, refund, dispute, and watch the score move. Agent instructions and the exam prompt are editable on the page.

How we built it

  • PayPal REST: Orders v2 (create, capture with a sandbox card, capture after owner approval via the return URL), Refunds, and Webhooks (CHECKOUT.ORDER.APPROVED, PAYMENT.CAPTURE.COMPLETED, PAYMENT.CAPTURE.REFUNDED, CUSTOMER.DISPUTE.CREATED), with signature verification and idempotent outcome recording.
  • Gemini 3.1 Flash-Lite via REST, four jobs: writes the exam, judges the cart against intent and classifies the failure, explains the grade in one sentence, and plays both demo shoppers with different system prompts. Nothing about the sloppy cart is hard-coded.
  • Cloudflare Workers + D1 + Static Assets with Hono and TypeScript. The dashboard is one HTML file. The ledger uses AG Grid Community.
  • Scoring, grading, tiers, and the decision table are deterministic code with unit tests.

Challenges

  • A credit bureau a model can be talked into is worthless. The whole demo is about an agent that got talked into something by a product description, so the grader and the decision table had to be code. Gemini only produces inputs and explanations.
  • Gemini over-penalized correct carts because the right cable's description carried the planted note. The fit judge now sees seller descriptions but is told an item that matches the request is good even if its description tried an injection the agent ignored.
  • Memorizable exams. A fixed catalog can be gamed, so Gemini generates a new one per test and a validator checks the traps really exist before the grader trusts it.
  • Sandbox quirks: the classic test card is rejected by the current sandbox, disputes cannot be opened from the API, and card capture needs Advanced Card Payments. Each has a documented fallback.

What we learned

Reputation is the missing primitive in agentic commerce, and it only works if the thing being measured cannot influence the measurement. Keep the model where it is irreplaceable (judging natural-language intent) and keep money decisions in a table anyone can read.

What's next

A shared standard: score portability across merchants, signed assessments, a dispute-weighted decay, and an issuer-side hook so a flagged agent cannot simply re-register.

Tools used

  • PayPal Developer Platform (sandbox): Orders v2, Refunds, Webhooks.
  • Google Gemini 3.1 Flash-Lite: exam generation, cart-vs-intent judging with failure classification, explanations, and the demo agents.
  • AG Grid Community: the assessment ledger.
  • Cloudflare Workers, D1, Static Assets, Hono, TypeScript.

Source and run instructions: https://github.com/sarchak/vouch · Live: https://vouch.shrikar84.workers.dev/dashboard

Built With

Share this project:

Updates

Submission history