Inspiration

What it does

How I built it

Challenges I ran into

Accomplishments that I'm proud of

What I learned

What's next for ReturnCheck

Inspiration

Camera rental shops protect themselves with a credit card hold for the full value of the kit. Camera Rentals NYC, for example, places a hold on uninsured renters and releases it "upon return and damage-free inspection" (https://www.camerarentalsnyc.com/rental-returns-policy/). The hold has to be sized before anyone knows the damage, and PayPal's authorization guide says an authorization "is valid for 29 days" with a 3-day honor period.

Letting AI bill directly fails too. The Autopian reported a Hertz customer "charged $440 for a scuff on a wheel" after AI scanner inspections (https://www.theautopian.com/customers-outraged-with-hertz-ai-scanners-they-say-automatically-charge-for-questionable-damages/, 2025-07-10). We wanted a flow where AI finds the scratch, but the renter and a person decide the charge.

What it does

A rental shop saves the renter's PayPal once at pickup (PayPal Vault v3) instead of holding their card. The renter returns the kit and leaves without waiting for an inspection. Then:

  1. Claude compares the check-out and check-in photos zone by zone and drafts priced line items with evidence crops. It also looks at earlier returns of the same asset, so an old barrel mark from a previous rental is not billed.
  2. Staff keep or drop each item and send the renter a link.
  3. The renter accepts or contests each line (with a photo and a note) from their phone.
  4. Claude can only request a charge on the saved PayPal token. The server sums the amount from the shop's rate card (the model has no amount argument), checks consent limits, item states and duplicates, and creates an approval card marked "Not executed until approved".
  5. Staff press Approve. Only then does PayPal Orders v2 charge the saved token. The receipt shows the order id, status, invoice_id, PayPal-Request-Id and the items PayPal returned; the server refuses an order whose amount, capture or invoice_id differs from what staff approved.

Also in the app: partial refunds through the same approval card, an invoice fallback when the payer removes the saved token, a read-only PayPal dispute watch, an evidence page, and an AG Studio fleet dashboard. The photo set in the demo is AI-generated photorealistic test images, not real customer photos.

How we built it

  • PayPal: Vault v3 (setup token, payment token), Orders v2 with payment_source.paypal.vault_id, and @paypal/agent-toolkit tools for orders, refunds, invoices (create_invoice, send_invoice) and disputes (list_disputes, get_dispute). packages/paypal-vault-tools adds six Vault v3 tools in the toolkit's tool shape (the toolkit has none).
  • AI: Claude Sonnet (claude-sonnet-5-5, Anthropic API) runs a 13-tool agent loop. Code owns amounts, allowed actions and the token/refund branch. The model decides whether a mark is new, how to treat an ambiguous zone, and which accepted items to request now versus which contested ones go to staff. People approve every money action. Photo comparison uses Claude vision with boxes mapped back to original pixels.
  • AG Grid: AG Studio powers the fleet dashboard at /studio: custom widgets (damage map with cross filter, recovery tile, queue), standard grid and charts, a shared theme, and a "Damage Analyst" Studio agent that delegates to Studio's built-in data and widget agents through a Claude adapter we wrote.
  • Hosting: the hosted demo runs live Claude Sonnet with per-IP-hour and per-day caps against the PayPal sandbox with a shared pre-saved demo payment token. When the API fails or a cap is hit, the action still finishes with a labelled recorded real Claude run or a labelled rule-based stand-in. The demo video was recorded on the hosted app.
  • Stack: React 19, Vite, Express 5, zod, Anthropic SDK and Claude Agent SDK. Judge mode gives each judge a private copy of rental R-1042 with its own claim id, invoice_id and PayPal-Request-Id.

Challenges we ran into

  • Keeping the AI out of the money path. Money tools have no amount argument and can only create an approval card; code prices from the rate card and checks duplicates and limits.
  • The Agent Toolkit has no Vault tools, so we wrote six in the toolkit's shape.
  • Real toolkit responses differed from the docs: create_invoice returned a link rather than an id and send_invoice returned a payer-view link, so we fixed createAndSendInvoice after running it against the sandbox. list_transactions answered 403, so the app does not use it.
  • Many judges share one demo payment token. We made sure the shared token is never deleted and checked two browser sessions charging the same token.
  • Honest evaluation: we have no real camera photo pairs yet, so we labelled what is seeded, synthetic or AI-generated instead of quoting a detection rate.

Accomplishments that we're proud of

Verified against the PayPal sandbox on 2026-11-12 (VERIFY.md): setup token, payment token, five merchant-initiated charges on one saved token plus two from the approval card in sandbox demo mode, all COMPLETED, two refunds from the approval card COMPLETED, Toolkit invoice create and send, and one shared token charged from two sessions.

With the real model (Sonnet): the inspection and settlement agent loops, the Studio agent, and vision on two synthetic PNGs. A 24-pair synthetic evaluation found 12 of 12 damaged pairs and flagged 0 of 12 clean pairs (4 with an old mark); the AI-generated photorealistic photo set matched 6 of 6 zones. These are drawings and generated images on easy input, not real camera photos and not a detection rate. 66 offline tests pass on doubles.

What we learned

  • Splitting the work (code owns amounts, the model owns judgment, people own approvals) makes an AI agent safe to put near a payment.
  • Saved-token charges work in the PayPal sandbox without the payer present, but the payer can remove the token in PayPal settings, so an invoice fallback is needed.
  • PayPal Seller Protection is not expected to cover a charge for damage found after an in-person hand-over (https://www.paypal.com/us/legalhub/paypal/seller-protection), so renter consent before the charge is the real risk control.
  • Claude's box coordinates are approximate (Anthropic vision docs), so staff can drag every box.

What's next for ReturnCheck

  1. Run real camera photo pairs through Claude and publish the result, misses included.
  2. Offer the six Vault tools to the PayPal Agent Toolkit, keeping money behind a human-approved card.
  3. Extend to other check-out and check-in rentals (tools, outdoor gear, cars), one AG Studio page per counter.

Built With

  • ag-grid
  • ag-studio
  • agent-toolkit
  • ai-agents
  • anthropic
  • claude
  • express.js
  • fintech
  • human-in-the-loop
  • node.js
  • orders-v2
  • payments
  • paypal
  • react
  • rentals
  • sandbox
  • vault
  • vision
  • vite
Share this project:

Updates

Submission history