Agent Flight Recorder

A flight recorder for AI-agent purchases: signed mandates, a deterministic guard before PayPal, and tamper-evident evidence you can verify offline when a purchase is disputed.

Comment

Inspiration

AI agents are starting to buy things on people's behalf. When one of those purchases is disputed, the dispute process has no record of what the agent was actually allowed to do. I wanted to build that record.

What it does

Agent Flight Recorder sits between an AI shopping agent and PayPal.

  1. Mandate. The person signs a narrow, expiring permission: budget, per-order cap, named merchants, allowed categories, order limit and an optional approval threshold. It is Ed25519-signed, so it cannot be quietly edited later.
  2. Guard. Plain deterministic code checks every proposed purchase before any PayPal order exists and answers ALLOW, NEEDS_APPROVAL or DENY with reason codes. No AI model is in the enforcement path: the model proposes, the guard decides. A denied purchase never reaches PayPal.
  3. Recorder. Every step goes into an append-only, SHA-256 hash-chained log. The log head can also be stamped by an independent RFC 3161 timestamp authority.
  4. Evidence. For a dispute, the signed mandate and the full log become an evidence bundle that anyone can verify offline: signature, every hash and the timestamp.

The AI in it

The shopping agent is a real AI model. In the live demo a small open model (Qwen2.5-1.5B-Instruct, Apache-2.0) runs locally through llama.cpp, reads a product catalogue and proposes a purchase. I deliberately poison the catalogue with hidden instructions. Across 24 live runs (4 injection wordings, 2 listings, 3 requests), the model proposed something outside the mandate 15 times. The guard denied all 15, and no PayPal order was created for any of them. A small model is a realistic worst case, which is exactly what the guard exists for.

How I built it

Python standard library plus cryptography (Ed25519), SQLite, and the PayPal sandbox REST APIs (OAuth, Orders v2, Customer Disputes, Webhooks). Money is integer minor units, never floats. A single-page dashboard shows the mandate, the guard's verdicts, the hash chain and a Verify button. An offline command-line verifier checks a downloaded bundle with no server and no network.

Challenges

  • Making the hash chain verifiable without the database, which led to bundles that carry the whole chain.
  • Being honest about limits: the log is tamper-evident, not tamper-proof. Removing the newest events is only detectable against a head hash or timestamp kept elsewhere, so the verifier reports how many events are not independently pinned.
  • Sandbox constraints: the default sandbox business account was not enabled for card payments, and capturing an order or opening a dispute needs a sandbox buyer to approve it in a browser.

Accomplishments (measured)

  • 146 automated tests.
  • Red-team suite: 27 of 27 attack scenarios blocked (prompt injection, inflated totals, lookalike merchants, replay, order splitting, forged mandates and more). Two in-scope cases are deliberately not blocked and are reported separately.
  • 209 of 209 single edits, deletions and swaps on an evidence bundle are caught by the offline verifier.
  • 5,000 seeded random cases: the guard matched an independent oracle every time.
  • Real PayPal sandbox orders created end to end through the guard and ledger.

Honest limits

  • The red-team scenarios and the oracle were written by the same author as the guard, so they are a regression suite, not independent testing. The live-model result is a small sample with my own injection wordings.
  • Dispute evidence submission is built against PayPal's documented Disputes API but has only been tested against a fake transport, not a live sandbox dispute. Webhook verification is likewise tested against a fake and not registered.
  • The default demo agent is scripted; the live scenario uses the real local model. The narration in the video is a synthesized voice.

What I learned

The interesting problem in agentic commerce is not getting an agent to pay; it is being able to show afterwards what it was allowed to do.

What's next

Run the evidence flow against a live sandbox dispute, per-tenant chains, and mandate templates for common purchases.

Built With

Share this project:

Updates

Submission history