Inspiration
A buyer pays with PayPal, the shop never hears about it, and the order sits as unpaid. It happens when a webhook is missed or the buyer closes the tab before the shop captures the payment. The merchant then gets an angry message and has to dig through PayPal by hand. The risky shortcut is to believe the message and mark the order paid. I wanted a tool that does the digging and never takes the buyer's word for it.
What it does
The merchant pastes the buyer's message into Recovery Desk. Then:
- A local AI model pulls the claims out of the message: order reference, amount, currency, PayPal IDs, date. Each claim must come with the exact words it was taken from, and code checks that those words are really in the message.
- The claims are treated as things to look up, not as facts. Code fetches the order and the capture from the PayPal sandbox and checks the merchant, the amount, the currency and the capture status.
- The desk gives one of eleven verdicts. Only two lead to an action:
- Paid at PayPal, shop not updated: propose marking the shop order paid.
- Approved by the buyer, never captured: propose capturing the payment, then marking it paid. Other verdicts explain why no recovery action is proposed, including cases that are already consistent.
- No recovery action executes until the merchant approves. On execution the desk re-checks PayPal, uses a stored idempotency key to guard retries, and verifies the result.
- Every PayPal response and shop read is stored as evidence with a SHA-256 hash, and a reply to the buyer is drafted from a template.
How I built it
- Python standard library only (sqlite3, http.server, urllib, decimal). pytest for tests. No framework, no paid service.
- PayPal Orders v2 and Payments v2 on the sandbox. The code refuses to run against anything but the sandbox.
- AI: Gemma 4 E4B (Q4_K_M) running locally in LM Studio, called on localhost with a JSON schema. A regex extractor runs alongside it; where the two disagree the field is dropped. If the model is unavailable the desk falls back to regex and says so on screen.
- A small demo shop (SQLite) stands in for a real store.
- A demo mode with simulated PayPal runs with one command and no credentials.
How PayPal and AI are used
For actionable payment recoveries, independent PayPal sandbox order and capture responses are the payment evidence. The demo substitutes simulated PayPal. Some conservative outcomes, such as unsupported claimed currency or missing IDs, stop before payment lookup. The only PayPal payment write supported is a capture with a stored PayPal-Request-Id. The AI reads messy buyer messages (other languages, amounts in words, IDs split across lines) so the lookup has something to work with. The AI has no tools and cannot change any payment or order.
What is real and what is simulated
- Real PayPal sandbox: on 4 Nov both recoveries (shop sync and capture) passed 7 of 7 checks. On 11 Nov the shop-sync recovery was run again with the local model doing the extraction. The capture recovery has not been re-run since the AI was added.
- Simulated: the 30-message evaluation and the demo mode use a fake PayPal. Every demo page says so.
- The buyer messages are written by me for testing. There are no real customers.
Results
- 138 automated tests pass, 1 skipped (the opt-in live sandbox test).
- 30 synthetic messages: no negative case led to a "paid" verdict or a proposed action, with or without the model. All four prompt-injection messages ended with no action.
- The model's gain is in reading messages: missed fields dropped from 17 (regex) to 2 (regex + model). Verdict accuracy was 29/30 with the model and 30/30 with regex alone, on a dataset I also used during development, so it is not an independent benchmark.
- Median extraction time with the model: about 2.9 seconds on a laptop GPU.
- Known failure, disclosed: a fake ID written inside a buyer message is still accepted as a claim (2 cases with the model, 1 with regex). The PayPal lookup then fails and nothing is proposed.
Challenges
- PayPal's capture call returns an order, not a capture, and an order marked COMPLETED is not proof of payment; the capture status has to be checked separately.
- The first model run returned nothing usable because the model spent its token budget thinking. Thinking is now switched off.
- An early evaluation looked better than it was because the "model" answers were built from the answer key. That was caught in review and relabelled as a wiring check only.
What I learned
Keep the model away from the decision. It is good at reading a message and bad at being trusted. Source quotes and independent PayPal checks keep extraction separate from payment decisions. In these finite tests, extraction errors produced zero false actionable verdicts; that is measured evidence, not a guarantee for every possible message.
Limits
One merchant, USD only, full captures only. Demo shop, not a WooCommerce or Shopify plugin. No disputes, no Transaction Search, no live webhooks. Sandbox prototype, not production software.
What's next
A real store adapter, verified webhooks, refunds and partial captures, and a larger message set that was not used during development.
AI tools used
- In the product: Gemma 4 E4B via LM Studio, local only.
- To build it: Claude (research, planning, task specs, code review), Google Antigravity (implementation), OpenAI Codex (coordination, test reproduction, sandbox runs, fixes). Details in docs/AI_DISCLOSURE.md in the repository.
- Video narration: offline Microsoft Windows Speech synthesis; edited real application footage with captions.
Run it
Source and complete instructions: https://github.com/ryuzaki1709/recovery-desk-ai
Run from the repository root: python3 -m recovery.web --demo (Windows: py -3 -m recovery.web --demo), then open http://127.0.0.1:8765. Default demo uses simulated PayPal and regex; optional local AI setup is documented in the README.
Demo video: https://youtu.be/LwkTuHY8wLc
