TL;DR: pay-per-pass for AI. PayPal holds the buyer's money, two independent AI inspectors check every result against the checklist the buyer agreed on, PayPal captures only what passed, and PayPal Payouts pays the studio its share. Every settlement becomes a signed receipt and a data point in a Quality Index that AI agents shop by.

Everything runs for real, with no mocks: PayPal sandbox and real AI models. So far: 16 settled orders, 64 images inspected (47 passed), $70.50 captured and $25.50 released from $96.00 held, $63.45 paid out to studios with PayPal Payouts, and 106 PayPal webhooks received, every one signature-verified.

How Earned settles an order

Inspiration

AI is the first product that gets sold before anyone knows whether it worked. You pay, and the model misspells a name, prints the wrong price or draws eight tulips instead of seven. Your options are a refund request, a dispute, or nothing.

We run a few small AI tools ourselves. Most of our paying customers arrive from chat assistants, on their phones, and they all hesitate at the same moment: paying before they have seen the result. Free previews help a little. A guarantee enforced by the payment itself would help a lot more.

The problem gets bigger with agents. When an AI agent buys from another AI, nobody is watching. "Verify, then settle" is the only rule that works there, and payments do not have that primitive yet. PayPal almost does: authorizations, partial captures and Payouts. So we built the missing half.

Who it is for

  • AI tool makers who sell generated work (cards, posters, product shots, logos, translations, code fixes) and lose buyers at the moment those buyers have to pay before seeing the result.
  • AI agents that buy that work for people. They need a rule that protects their owner without a human checking every result.

What it does

  1. Agree the checklist. An LLM turns the buyer's words into 3–6 objectively checkable items: exact text, counts, colours, exclusions. Factual items decide payment; style wishes are inspected and reported but never billed. The buyer edits it, and its SHA-256 hash is written into the PayPal order's custom_id, so nobody can move the goalposts later.
  2. PayPal holds the money. An Orders v2 AUTHORIZE order. The studio knows the money is real; the buyer knows it hasn't moved.
  3. Two AI inspectors check every result. Two different model families transcribe any text character by character and judge each item, quoting evidence. If they disagree on a required item, a stronger third model decides it.
  4. PayPal captures only what passed. 3 of 4 passed? One partial capture for exactly 3 × $1.50 with final_capture: true, and the rest of the hold is released. Nothing passed? The authorization is voided.
  5. PayPal pays the studio. Right after settlement, PayPal Payouts sends the studio its share minus Earned's 10% fee.
  6. Appeals and receipts. A buyer can appeal a charged item once; if a stronger model upholds it, PayPal refunds that item immediately and the studio's share is recovered from its next payout. Every settled order gets an Ed25519-signed receipt that anyone can verify, including in the browser.

Three ways in: a web studio; a shopping agent that ranks studios, drafts the checklist and opens the order (but cannot approve the payment); and an MCP server, so any AI agent can shop this way. Agents can open orders; only a human can approve the PayPal hold.

The Quality Index ranks four demo AI studios (each running a different image model) by their pass rate from settled PayPal orders, using the Wilson lower bound so two lucky orders can't top the table.

What makes it different

  • It inspects before anyone pays, instead of judging a dispute afterwards. Escrow and dispute tools step in once a deal has gone wrong and decide who is right. Earned checks every delivery against the agreed checklist before money moves, so a fair charge never depends on the buyer complaining.
  • It charges per item, not all-or-nothing. Most AI output is "3 of 4 good"; one partial capture charges for exactly the 3.
  • The contract is machine-checkable and fixed up front. Facts decide payment, taste never does, and the hash is in the PayPal order before work starts.
  • No model judges alone. Two model families plus a tiebreaker, with every vote and its evidence stored.
  • Its reputation is made of money. The Quality Index counts only orders that settled through PayPal, so a studio cannot buy a better rank without paying for real passes.
  • Both sides get paid properly. Buyers pay only for passes, studios are paid with PayPal Payouts, and refunds after a payout are netted out of the studio's next one.

Why studios would opt in

  • A failed item costs them at most 3% of the price. In this demo an image costs the studio $0.007–0.045 to make (measured from our gateway bills) and sells for $1.50. In return, buyers who would never pay up front will try them.
  • Demand follows quality. Agents choose studios by money-backed pass rate, so the studio that delivers gets the next order without buying ads.
  • Little left to dispute. Buyers were never charged for the work that failed, and every charge comes with quoted evidence and a signed receipt.

How we built it

  • PayPal: Orders v2 (intent: AUTHORIZE, items, custom_id contract hash, experience_context return/cancel URLs, PayPal-Request-Id idempotency), orders/{id}/authorize, authorizations/{id}/capture with final_capture, authorizations/{id}/void, captures/{id}/refund, Payouts (/v1/payments/payouts, one batch per order, polled until final), and webhooks (CHECKOUT.ORDER.APPROVED, PAYMENT.AUTHORIZATION.*, PAYMENT.CAPTURE.*, PAYMENT.PAYOUTSBATCH.*, PAYMENT.PAYOUTS-ITEM.*) verified with verify-webhook-signature and shown on each order's PayPal trail. If the buyer closes the tab after approving, the webhook finishes the job.
  • AI: gpt-6-luna drafts checklists; gemini-3.8-flash and gpt-6-luna are the two independent vision inspectors; gemini-3.1-pro-preview breaks ties and hears appeals; gemini-3.8-flash with tool calling runs the shopping agent; four image models (qwen-image-2.0, z-image-turbo, flux-3-image-klein, gpt-image-2.6-flare) power the demo studios. Everything goes through one OpenAI-compatible gateway, so providers are swappable.
  • Stack: Cloudflare Workers, D1, R2, TypeScript, no frontend framework. The order is a small state machine (authorized → generating → judging → settling → settled → studio paid) that any caller can advance (the order page, an MCP client polling status, or a cron trigger), with a lease in D1 so steps never run twice.
  • Safety and cost: a spend ledger with a hard cap and a daily slice for the public demo, per-visitor limits, and a dedicated, capped API key.
  • Written with help from an AI coding assistant (Claude Code).

Challenges we ran into

  • Judges that were too literal. Our first inspectors failed a poster because a "·" separator became a line break, and once failed a card that was "for my mum" because the name on it was Linda. We fixed it at the source: separators are layout, context isn't a requirement, and only factual items move money.
  • Consistency. One inspector passed an image another failed for the same reason. Two model families plus a tiebreaker, with every vote stored, made verdicts stable and auditable.
  • Making money and AI line up. Partial capture, payouts and refunds must match the verdicts exactly, survive retries and never double-pay: idempotency keys on every PayPal call, the split booked before the payout is sent, one state transition per poll, and receipts re-signed whenever the money changes.

Accomplishments that we're proud of

  • A real end-to-end flow on the PayPal sandbox: hold, inspect, partial capture or void, payout to the studio, refund on appeal, verified webhooks, signed receipt.
  • An agent that can shop but cannot pay, which is how agentic commerce should work.
  • A quality ranking built only from money that actually moved.

What we learned

Most AI results are not all bad; they are "3 of 4 good". Payments need a primitive for that, and PayPal's authorize + partial capture is it. The hard part isn't the payment; it's writing the contract so a machine can check it.

What's next

  • Today: outcome pricing as a drop-in for any AI tool. One integration turns pay-first into pay-per-pass.
  • Next: a credit score for every AI. Benchmarks get gamed and reviews get faked; pass rates from settled money are much harder to fake. Agents read the index before they buy.
  • Endgame: the settlement layer of the agent economy. When agents hire agents, whoever verifies outcomes decides where the money goes. Every payment becomes a vote: money flows to AI that works, and AI that doesn't stops getting paid.
  • Production path: PayPal multiparty (platform) so each studio is its own merchant and funds never sit with Earned; milestone holds for longer jobs; human review behind appeals.

Built With

  • ai-agents
  • cloudflare-d1
  • cloudflare-r2
  • cloudflare-workers
  • gemini
  • llm
  • mcp
  • openai
  • paypal
  • paypal-orders-api
  • paypal-payouts
  • paypal-webhooks
  • typescript
Share this project:

Updates

Submission history