Holdfast

A PayPal deposit hold settled on evidence: Gemini spots new damage in pickup and return photos, the shop's price list prices it, and PayPal captures only what both sides approve.

Comment

Inspiration

Small rental shops (camera gear, bikes, audio, outdoor kit) live on deposits, and deposits cause fights. The shop says the item came back damaged; the renter says it was already like that. The usual outcome is a surprise charge, a chargeback, or a shop that gives up and absorbs the loss. Neither side has a shared record of what changed, or a fair, visible way to settle it.

What it does

Holdfast turns a rental deposit into an evidence-backed settlement.

  1. Hold, not charge. At booking, the renter approves a PayPal order with intent: AUTHORIZE. The deposit is held as a PayPal authorization, and nothing is captured.
  2. Photos at pickup. Staff take six guided photos. Gemini records the item's existing wear as a baseline, and the renter acknowledges it from a signed link.
  3. Compare at return. Staff take the same six angles. Gemini compares each pair and proposes only what is new: type, severity, location, confidence and a quoted reason. Wear already in the baseline is not charged.
  4. The shop's price list sets the price. A deterministic pricer maps each finding to the shop's own price list. The AI never sets a price.
  5. People decide. Staff confirm, re-price or drop each line. The renter sees every photo and price, then accepts or contests each line with a reason.
  6. PayPal settles exactly that. Only after both approvals does Holdfast capture the agreed amount and void the rest of the authorization:

$$\text{captured} = \sum \text{approved lines}, \qquad \text{voided} = \text{hold} - \text{captured}$$

A clean return voids the whole authorization. Both sides see the same receipt, with PayPal's own IDs and statuses, each confirmed by a signature-verified webhook.

How I built it

PayPal is the money layer.

  • Orders v2 creates and authorizes the deposit.
  • Payments v2 captures part of it with final_capture: false, then voids the remainder. Capture-then-refund is a tested fallback mode.
  • Every write carries a PayPal-Request-Id, so a retry can't capture twice, and settlement runs under a database lock.
  • Webhooks (PAYMENT.AUTHORIZATION.CREATED, PAYMENT.CAPTURE.COMPLETED, PAYMENT.AUTHORIZATION.VOIDED) are verified with PayPal's verify-signature API before anything is recorded.
  • The PayPal agent toolkit doesn't expose authorize, partial capture or void, so Holdfast calls the REST APIs directly.

Gemini is the evidence layer.

  • gemini-3.7-flash on Vertex AI returns findings against a strict JSON schema, and the output is validated with Zod.
  • A baseline pass records existing wear; a return pass compares each photo pair against it.
  • Uncertain findings go to a "needs your eyes" group instead of the bill.
  • A daily call cap protects the demo.

The rest of the stack: Next.js 16 and TypeScript (strict), Tailwind CSS and shadcn/ui, Firestore, and a private Cloud Storage bucket, deployed on Google Cloud Run. Money is integer cents everywhere. The pricer, the rental state machine and the settlement planner are pure functions covered by 653 unit tests.

Challenges I ran into

Proving PayPal's behaviour before building on it. Before writing any settlement code, I ran a Sandbox spike:

  • partial capture plus void works;
  • after final_capture: true, a void fails with PREVIOUSLY_CAPTURED, and the authorization reports CAPTURED even though only $38 of $200 was taken;
  • retries with the same request ID return the same capture.

Those results chose the settlement mode and the wording. Holdfast says captured and voided, never refunded or released, because those are the statuses PayPal actually returns.

False charges destroy trust. The hardest cases are clean returns in different light, at a different angle or on a different background. I built an evaluation set from real photos of my own gear and measured the false-charge rate, not just accuracy:

  • all 3 cases of new damage were found;
  • both cases of pre-existing wear were correctly left uncharged;
  • 1 of 5 clean returns got a proposed charge.

That one false charge is exactly why a person confirms every line and the renter can contest it. It's a small set (8 real cases), so I read it as evidence per case, not as an accuracy claim.

Keeping the AI away from the money. No code path lets model output set a price or start a transaction. Settlement code itself checks that the staff and renter approvals are recorded.

Accomplishments that I'm proud of

  • The whole flow is real: PayPal Sandbox authorizations, captures and voids, plus live Gemini inspections on real photos. Every PayPal ID shown in the demo video was re-checked against PayPal's API afterwards.
  • A clean return in different light comes back "No new damage", and the whole hold is voided.
  • A renter who disagrees has a real path: the contested line is reviewed, dropped, and the hold voided.

What I learned

Authorization lifecycles are subtle: a single flag, final_capture, decides whether the rest of a hold can still be voided. Vision models are strong at spotting missing parts and dents, and weakest when only the lighting or background changes. Designing for disagreement (visible evidence, contest rounds, human gates) mattered as much as the model's accuracy.

What's next

  • A larger evaluation set across more item types.
  • A view of PayPal disputes in the timeline.
  • A pilot with a real rental shop, which would need a live PayPal account and its own review.

Disclosures

  • PayPal Sandbox only. No real money moves.
  • Real photos only. Gemini findings come from live calls on real photos of my own items; no AI-generated or AI-edited images are used.
  • Video voice. The demo video's narration is an AI-generated voice (Google Cloud Text-to-Speech).
  • New work. Holdfast was built from scratch during the hackathon, starting Oct 27, 2026.

Built With

Share this project:

Updates

Submission history