Refund Desk

Inspiration

PayPal's agent tooling lets a model call refunds directly. That is what makes AI after-sales support possible for a small merchant. It also means one persuasive chat message ("it arrived broken, refund me 300") can turn into real money leaving the account. A merchant today has two choices: answer every refund request by hand, or let the model move money with nothing between its words and the payment. We wanted a third choice: let the AI do the talking, and let the payment record decide.

What it does

Refund Desk has two panes.

Customer pane. The customer proves they own an order (order id plus the payer email PayPal has on file) and describes the problem in a chat. The AI agent looks up the order through PayPal and drafts a refund proposal. It cannot send a refund itself.

The gate. Every proposal goes through seven checks. Each check reads PayPal (the capture, its refunds, its disputes) and the local database. It never reads the model's reasoning, its draft reply or the chat text.

1. The capture is completed or partially refunded.
2. The capture belongs to the order the customer proved.
3. The amount is well formed and in the capture's currency.
4. The amount fits within what was captured, minus refunds PayPal lists, minus refunds this app has in flight.
5. The capture is inside the merchant's refund window.
6. The capture has no open dispute.
7. Any fail rejects. Anything the gate cannot determine (a timeout, an unknown status) goes to manual review and never sends money. Otherwise small refunds execute automatically and larger ones wait for the merchant.

Merchant pane. The merchant sees the approval queue and approves or declines with one click. An approval runs the whole gate again against fresh PayPal data. Every proposal has a receipt showing each check, each decision and each PayPal call.

How we built it

• Python, FastAPI and httpx, with SQLite for proposals, receipts and chat.
• PayPal REST APIs in the sandbox: Orders v2 for order lookup, Payments v2 for captures and refunds, and the disputes API for check 6.
• Refunds carry a stored PayPal-Request-Id, so a retry after an unclear answer cannot refund twice. Proposals on the same capture are serialized, so two of them cannot both pass check 4 against the same remaining amount.
• The chat agent works with any model behind a small adapter. The demo uses qwen3.5 running locally through Ollama, so chat text never leaves the merchant's machine. The app refuses to start if the local model URL is not a loopback address. An Anthropic adapter is included. A scripted demo agent (not an LLM) is included for running without any model.
• 703 automated tests run offline against a fake PayPal and fake model replies, plus an optional live sandbox smoke test and browser UI behaviour tests.

Challenges we ran into

• "Not sure" must never mean "yes". Every PayPal call can time out or return something unexpected. We made each check three-valued (pass, fail, undeterminable) and routed every undeterminable result to a person instead of guessing.
• Retries that do not double-refund. A refund request can time out after PayPal has already executed it. The stored request id plus a reconcile step lets the app find out what happened before trying again.
• Small local models invent things. In our tests one 8B model made up a capture id instead of looking up the order. qwen3.5 looked it up correctly. Even so, the gate checks every id against PayPal, so an invented id is rejected either way.

Accomplishments that we're proud of

• The safety of a refund does not depend on the model behaving. A model that is fooled, or simply wrong, can only produce a proposal, and the gate judges that proposal on PayPal's numbers.
• The whole loop runs end to end against the real PayPal sandbox with a free local model.
• Every decision is explainable after the fact from its receipt.

What we learned

The model is good at the conversation: understanding the complaint, finding the order, explaining the outcome. It is the wrong place to put the rules about money. Keeping those two jobs apart made both simpler to build and to test.

What's next for Refund Desk

• Disputes and subscription cancellations through the same gate.
• Per-customer and per-day limits.
• A packaged install for small merchants.

How AI is used (for the judges)

• What the AI does: reads the customer's message, calls the order lookup tool, decides what to propose, and writes the customer-facing reply.
• What the AI cannot do: send money. Its only effect is a proposal, and the gate decides on PayPal data alone.
• What runs in the demo: qwen3.5 (about 9.7B parameters) on a laptop through Ollama. One refund turn took about 30 seconds.
• The scripted demo agent is not an AI. It proposes exactly the amount the customer typed. We use it to show the gate stopping a proposal that a fooled model would make.

Built With

• fastapi
• httpx
• ollama
• paypal-rest-api
• paypal-sandbox
• python
• qwen
• sqlite