-
-
proofpay
What's next for ProofPay: Pay AI Agents Only for Proven Work
Inspiration
AI agents can now do real work, and soon they will be paid for it. But an agent can say a task is done when the result doesn't work, and asking a person to approve every routine payment defeats the point of automation. We wanted payment to depend on proof the agent did not write itself, inside limits the buyer chose.
What it does
A buyer commissions a $75 repair of a café's broken booking form, with five agreed checks. Buyer and developer approve the same version, and the buyer authorizes the amount through PayPal and sets per-payment and monthly limits.
A repair agent reads the broken handler and asks a model for a patch. Real Chromium then tests every patch against the same five checks, which the agent cannot change. A patch that only displays a success message without sending the reservation fails. Only a passing patch becomes a delivery, identified by the SHA-256 of its HTML.
Payment follows the evidence: the buyer's standing rules capture the existing PayPal authorization only when the agreement, checks, scope and budget all match. A failed or inconclusive check blocks payment.
Afterwards the buyer can ask "why did this payment happen?" Decision replay shows the exact inputs saved at capture time, a policy lab reruns the same evaluator on copies (lower the limit, fail a check) without touching the record, and a proof bundle exports everything with a checksum.
If something is wrong, the buyer reports a problem: automatic payments pause, the evidence is preserved, and the seller can refund fully or partially against the original capture.
How we built it
- PayPal: Orders and Payments sandbox APIs for authorization, conditional capture, void, full/partial refund and reconciliation, with request IDs so a retry cannot pay twice. Webhooks are signature-verified.
- AI: OpenAI structured outputs write the booking-handler patch. The model never decides payment eligibility.
- Verification: a Playwright worker runs the stored artifact in a sandboxed origin and keeps screenshots as evidence.
- App: React, Vite, Express, PostgreSQL (PGlite locally), TypeScript. 80 unit/API tests and 7 real-browser tests.
Challenges
- A patch can look fine on screen but never send the reservation, so the check requires a successful POST to the approved endpoint and the exact visible text. Failures now tell the agent which half failed so its second attempt can fix it.
- A capture or refund can time out with an unknown outcome. Amounts stay reserved until PayPal confirms, and retries reuse the same request ID.
- Keeping the agent away from its own finish line: checks, price and rules live in a versioned agreement that only the two people can approve.
What we learned
Verification has to be independent of the agent and agreed before the work starts, or it is not evidence. Automatic payment is only acceptable when the limits are the buyer's, exceptions return to a person, and a refund path exists.
Honest status
The guided demo uses real Chromium checks with an authored broken/fixed page and simulated payments. PayPal sandbox sign-in and a buyer-approved $75 authorization have succeeded. The model-backed repair is built and tested with a mocked model; a live model run, sandbox capture/refund and hosted deployment are not yet verified. Full details are in the repository's validation record.
What's next
More task types with machine-checkable results, multiple sellers, PayPal Disputes integration, and PayPal's agentic commerce services so outside agents can take and settle tasks directly.
To run it: clone the repo, then npm ci && npm run browser:install && npm run dev, open http://127.0.0.1:5173 and click Start interactive demo. No keys or accounts needed.
Built With
- chrome
- docker
- express.js
- node.js
- openai
- paypal
- paypal-orders-api
- paypal-payments-api
- pglite
- playwright
- postgresql
- react
- render
- typescript
- vite
- zod
