Recovery Proof - Checkout Recovery Flight Recorder

Unknown is not failed. Source-backed recovery decisions for PayPal checkout agents.

Comment

Inspiration

When an AI shopping agent sends a checkout request and loses the HTTP response, it faces a dangerous ambiguity: did the provider create the order, or did the request never arrive? Treating UNKNOWN as FAILED can produce an unnecessary second order. Treating a missing response as successful can be just as misleading.

Recovery Proof asks a different question: what can the agent actually prove before anyone authorizes another action?

What it does

Recovery Proof is an evidence-first checkout recovery workbench. It reconstructs the merchant's original order intent and request journal, compares that record with any observed provider state, and refuses to promote uncertainty into a financial decision.

The interactive workbench lets a judge explore 13 clearly labeled fictional incidents, including a lost response before or after a simulated commit, an aged idempotency key, a changed outbound amount, conflicting order IDs, a quote-to-order price mismatch, another active order for the same cart, and an unrelated-cart negative control. The system produces a bounded HOLD, review, or read-only verification recommendation with its supporting facts. It never automatically creates, captures, refunds, or retries a payment.

A genuine Qwen3-4B local AI investigator explains a selected evidence discrepancy, cites the relevant source records, and asks a focused verification question. An independent evidence reviewer rejects unsupported assertions. The AI does not get to override the deterministic decision.

PayPal + AI integration

We separately used the real PayPal Developer Sandbox Orders v2 API with a fictional USD 19.00 test order: Sandbox OAuth returned HTTP 200, Create Order returned HTTP 201, and GET Order returned HTTP 200 for that same CREATED order.

Recovery Proof can validate a locally redacted record of that authenticated Sandbox flow and have the real local AI explain what the recorded evidence proves, and what it does not. This AI explanation was exercised without any new PayPal call or money movement.

Important boundary: the authenticated Sandbox Create/GET was a normal API smoke test. It was not a real network timeout or a demonstrated real-world recovery. The 13 timeout and duplication scenarios are explicitly synthetic.

How we built it

The policy, local HTTP server, source checks, idempotency safeguards, and tests are written in Python using the standard library. The browser workbench uses original HTML, CSS, and JavaScript. Local Qwen3-4B-Instruct-2507 is served through llama.cpp on loopback; its response is schema-checked and independently evaluated against the evidence. The PayPal adapter is restricted to the Developer Sandbox and keeps credentials outside the source repository.

The public MIT-licensed source repository passed 228 automated tests in an independent clean checkout. We also exercised the guided workflow in Chrome at desktop and mobile sizes. The local model has produced independently accepted explanations of the authenticated historical Sandbox record and the fictional incidents; model response time can vary with local hardware.

Challenges and lessons

A retained idempotency key is not proof that a provider processed an order. A 404 lookup does not prove that creation never happened. An order in APPROVED state does not prove money was captured. And model-generated prose can sound confident while contradicting the request journal.

The hardest work was making those differences visible and independently testable, rather than masking them with a plausible AI summary. We learned to preserve evidence provenance, label simulation honestly, and keep human authorization outside the language model.

Why it matters

Commerce-agent developers and merchants need a trustworthy explanation after an ambiguous checkout event, not another blind retry. Recovery Proof demonstrates a narrow safety capability: source-backed reasoning that can justify stopping until the missing evidence is found. We have not claimed measured chargeback savings, recovered funds, or production merchant adoption.

Try the working build

The source repository includes a judge quickstart, reproducible local server, all fictional fixtures, a three-step browser walkthrough, tests, and optional instructions for a judge-owned free PayPal Developer Sandbox and local AI model. No paid hosting is needed to run the synthetic workbench. The real model and a judge-owned Sandbox account are optional separate setup steps to reproduce those integrations.

Built with

PayPal Developer Sandbox; PayPal Orders v2 REST API; Python; JavaScript; HTML/CSS; Qwen3-4B-Instruct-2507; llama.cpp.

What's next

Extend the evidence connector to read real provider-side audit trails for a specifically authorized Sandbox failure exercise, improve local model latency, and evaluate recovery decisions under more adversarial incident conditions. Human approval and no automatic financial writes remain non-negotiable.

Built With

  • ai-agent
  • human-in-the-loop
  • llama-cpp
  • paypal-orders-v2
  • paypal-sandbox
  • python
  • qwen3
Share this project:

Updates

Submission history