Inspiration

A parcel has a tracking number. When a buyer says it never arrived, the seller shows the scan at the door. Freelance and agency work has no tracking number. When a client accepts a website, pays, and later disputes the charge with their bank, the freelancer has emails, a link and their word.

This happens. In a Datos Insights survey of 840 merchants for Visa (2025), 21 percent of merchants' chargebacks were first-party fraud: customers disputing purchases they made. And the other side of the same relationship is late money: 56 percent of US small businesses are owed money on unpaid invoices (Intuit QuickBooks, 2025, 2,487 businesses).

I work at a small agency that is paid per milestone, when the client accepts the work. So I built the thing that would let a two-person studio prove, to a client and to a bank, that the work was delivered and accepted.

What it does

Signoff builds a signed acceptance record for each milestone, then uses it.

  1. Agree what done means. The freelancer pastes the scope. Google Gemini drafts milestones with checkable acceptance criteria, each quoting the sentence of the scope behind it, and the code checks the sentence is there. Lines of taste such as "modern" or "mobile friendly" are kept as criteria a person judges. They are what clients argue about later, so they belong in the record.
  2. Hold the money. The client opens a link, with no account, and places a card hold through the Braintree Drop-in. It is an authorization: held, not charged. Only a transaction Braintree returns as authorized counts as a hold, and the page warns when a hold would end before the milestone is due.
  3. Check the delivery. The checker reads the delivered pages and files with read-only tools and marks each criterion met, not met or needs a person. Every "met" must carry a quote that the code finds word for word in what a visitor can see, with the numbers and names the criterion states. Text hidden from visitors never counts, and instructions aimed at the AI are listed as tampering. When the code downgrades a verdict, the report says why in place of the model's sentence.
  4. Sign off. The client reads the report and signs off by typing their name, asks for changes with a note drafted from the failures, or declines, which voids the hold. Sign-off captures the hold, in full or in part, and writes the acceptance record: criteria, verdicts, the evidence the client saw, who signed and when, signed with HMAC-SHA256.
  5. Answer the chargeback. If the client's bank opens a chargeback after sign-off, Signoff drafts the evidence from that signed record. Nothing is sent until the freelancer approves it; then it goes to Braintree's Disputes API, and the outcome comes back on the page.

A freelancer's coding agent can work the freelancer's side over MCP: list milestones, submit a delivery, read the evidence report. It has no payment tools.

What it does not do. The hold sits in the freelancer's own Braintree account, so a freelancer could still capture it from the Braintree control panel. Card holds end after about a week (the sandbox hold in the video expires 7 days 23 hours after it was placed); the client can renew one, but Braintree cannot extend it. The production fix is a neutral platform account that holds the right to capture.

How I built it

  • Braintree, a PayPal service. The contest rules list Braintree as a PayPal service. I build from Pakistan, where PayPal does not open accounts, so the Braintree sandbox was the PayPal service I could use end to end, and it gives this product what it needs: authorizations that hold money without charging, partial capture, voids and a Disputes API. transaction.sale with submitForSettlement: false places the hold. submitForSettlement captures it in full or in part. transaction.void releases it after re-reading that it is still authorized. Webhooks are verified with webhookNotification.parse and the transaction or dispute is re-read from Braintree. Reconcile compares local state with transaction.find and dispute.search. Dispute defence uses dispute.addTextEvidence and dispute.finalize. The PayPal option in the Drop-in is built behind a flag and needs a linked PayPal sandbox account, which I cannot open, so the demo takes cards.
  • Money moves once. Every money move goes through one state machine. Release and void claim the milestone with a compare-and-set before calling Braintree, so a sign-off and a decline racing each other cannot both reach it.
  • Google Gemini through the Vercel AI SDK. A check reads every link and file in parallel, explores further with tools only when there is more to read, then judges all criteria in one structured call. The evidence rule in code decides what survives.
  • Measured. pnpm eval runs 20 deliveries and 80 criteria with known answers: prompt injection in six forms counted on its own line, every hiding method, a swapped phone number and date, a hollow PDF, lines of taste, an honestly complete page and an honestly partial one. On the final run of 10 November, with gemini-3.5-flash-lite first in the model chain: 79 of 80. A model answered 19 of the 20 deliveries and got all 77 of their criteria right; on the last one every model was rate-limited, the scripted fallback answered and missed one. Injection: 33 of 34 criteria on 9 deliveries, with the instruction flagged on 8 of 9 reports; the unflagged one was written in Spanish. A keyword checker with no AI gets 53 of 77.
  • AG Grid and AG Charts, community editions. The project page has a money grid with one row per milestone (agreed, held, released, voided, disputed, Braintree transaction and dispute ids) and a pinned totals row, an AG Charts chart of the same money, and an audit timeline grid with date and text filters, kind chips, id columns and CSV export. Clicking a milestone filters the timeline to it.
  • MCP. /api/mcp gives a coding agent three tools on one project, with revocable hashed tokens. A lint rule stops any file in the AI code from importing the payment rail.
  • App. Next.js 16, TypeScript strict, Tailwind with shadcn/ui, Drizzle on libSQL, hosted on Vercel with Turso. Unit and integration tests: 215 pass, and 13 live tests against the real Braintree sandbox and Gemini are skipped unless opted in. Playwright runs the whole loop in a browser, including at 390 px. GitHub Actions runs both on every push.

Challenges I ran into

  • No PayPal account. PayPal does not open accounts in Pakistan. The Braintree sandbox does, so Signoff runs there, and the PayPal option waits behind a flag for a linked PayPal sandbox account.
  • A sandbox chargeback after sign-off. Braintree's sandbox opens a chargeback only for its dispute test card, and it charges that card at once, so it cannot be a hold. After sign-off, a labelled sandbox button makes a stand-in charge with that card and ties its real dispute to the signed milestone. The sandbox rules the dispute from a test value in the evidence, and the page says so.
  • Making the AI prove what it says. Early versions could "prove" a criterion by quoting the requirement back, a page title, or hidden text. Each is now a named reason a quote is removed, with a test.
  • Honest numbers. Growing the eval found two misses in my own evidence rule, not in the model: a short price line dropped as too short, and a list quoted without its heading. I fixed the rule and kept the cases.
  • A public demo on a free tier. Gemini's free tier allows about 500 calls a day on each lite model. Checks use one model call, and daily caps per workspace, per visitor and per server keep the live model available.

Accomplishments that I'm proud of

  • A real Braintree sandbox chargeback, opened after sign-off and answered with evidence from the record the client signed, through to the outcome.
  • The cheat delivery is caught on screen: hidden text, an instruction aimed at the AI and a pasted requirement listed as tampering, and every removed quote shown with its reason.
  • 77 of 77 on the eval deliveries a model answered, 79 of 80 overall, with prompt injection counted apart and every miss reported as a miss.

What I learned

  • An AI verdict is only as good as the evidence you can check. Requiring word-for-word quotes, and checking them in code, turned the model from a judge into a witness.
  • The safest way to keep an agent from moving money is to never give it the tool.
  • A dispute is decided on records. The record is worth building before anyone needs it.

What's next for Signoff

  • Hold the right to capture on a neutral platform account, so neither side can move the money alone.
  • PayPal holds through Braintree once a PayPal sandbox account can be linked.
  • Render delivered pages in a headless browser, to catch text hidden by scripts.
  • Criteria for design files and pull requests with their test runs.

Built With

  • ag-charts
  • ag-grid
  • braintree
  • drizzle-orm
  • github-actions
  • google-gemini
  • libsql
  • model-context-protocol
  • next.js
  • playwright
  • react
  • shadcn-ui
  • tailwindcss
  • turso
  • typescript
  • vercel
  • vercel-ai-sdk
  • vitest
  • zod
Share this project:

Updates

Submission history