RefundDesk

An AI refund desk that proposes, a calculator that limits, and an owner who approves, on PayPal sandbox.

Comment

Inspiration

Small shops get refund requests as messy emails: "two plates arrived broken, refund me $140". The owner has to find the order, read their own policy, work out what is actually owed, and then refund the right capture without paying twice. AI can read the email quickly, but it should not be the one moving money.

What it does

RefundDesk turns a refund complaint into a policy-checked proposal, an owner decision and an exactly reconciled PayPal ledger.

  1. AI triage, read-only. The model gets the sender's candidate orders and the merchant's policy clauses, and can call read-only lookup tools for more; it returns a strict-schema proposal: the matching order, intent, damaged units, cited clauses with verbatim evidence spans, and a reply draft. It cannot approve, confirm facts or call PayPal.
  2. A calculator sets the ceiling. Allowed amounts are computed in integer cents from the policy and the capture's remaining balance. In the demo, Priya asks for $140; the policy allows $72 (two broken plates plus shipping), and the drawer shows both numbers side by side.
  3. The owner confirms facts and approves. Proposed facts stay separate from owner-confirmed ones. Approval binds to the reviewed revision and the displayed amount; the server rejects anything above the allowance, including after a concurrent edit.
  4. PayPal refund with lifecycle and recovery. Refunds use persisted PayPal-Request-Id values, so a retry never pays twice. Uncertain outcomes get a Check PayPal action; reconciliation matches exact receipts, amounts and currencies, and blocks refunds it cannot account for.
  5. An AG Grid review queue. Saved views (Needs attention, Risky, Ready to approve, Refunded), floating filters, pinned totals over filtered rows, batch selection with model-vs-allowance totals, and formula-safe CSV export. All Community edition.

How we built it

  • React + AG Grid Community on Vite; Hono serves the API and the production UI; SQLite stores cases, triage traces, orders, refund attempts and ledger rows.
  • The AI SDK calls any OpenAI-compatible provider with a strict proposal schema; a codex-cli provider sends the same prompt and schema through the local codex exec, for people whose model access is a Codex login. Every triage stores provider, model, latency, attempts, any tool calls, and cited spans.
  • PayPal REST, all on api-m.sandbox.paypal.com: OAuth, Orders (create, capture, get), Captures, Refunds, Transaction Search for recovering uncertain seed captures, and webhook signature verification.
  • A zero-key replay demo runs the same service, calculator and approval checks with a network-free payment adapter. Payment records keep their real sandbox IDs; new refunds in replay are deterministic simulations, and a permanent badge says so.
  • 131 tests, typecheck and production build in CI.
  • Built with substantial help from AI coding agents (Anthropic Claude Code and OpenAI Codex), which also wrote the 30 held-out evaluation cases and reviewed the code in several rounds.

Evaluation

30 held-out customer messages, written separately and never used to tune the prompt, run through the real triage path. Each case gets its own in-memory copy of the seed book, and no payment calls are made.

  • Live model (gpt-6.1-sol through the Codex CLI): 21/30 cases pass every check; intent 27/27; 0 over-asks; 0 unsafe allowances; 0 unsafe executions.
  • Four of the nine misses are cases where the message names an order that belongs to a different customer. The model declined to match it and asked for a human, e.g. "FC-1023 belongs to Iris Park, with a different email from sender Ben Walsh."
  • The offline keyword baseline passes 7/30 on the same cases.
  • Separately, every recorded model output is replayed through the real approval service: amounts above the calculator's allowance are rejected before any payment call.

Challenges

  • Keeping the model useful without letting it near money: tools are read-only, and facts it proposes never count until the owner confirms them.
  • Refund calls whose response is lost. We recover them by request ID and receipt lookup, and tested this against the live sandbox.
  • Free-tier AI quotas are per day, not just per minute. The evaluation saves after every case and resumes only when the model, provider, prompt, schema, policies, seed data and message text are all unchanged; in the end we ran it through the Codex CLI instead.

What's next

Inbox integration, multi-owner roles, a deployed webhook endpoint, and a production (non-sandbox) mode after a proper review.

Try it

npm ci && npm run demo

Open http://localhost:15481 — no PayPal keys, AI keys or password needed on loopback. Choose Risky, open a case, expand How the AI got here, confirm the facts, and approve.

Links

Built With

Share this project:

Updates

Submission history