Inspiration
A payment provider can report a successful Capture while the merchant's own order remains pending. For developers, the difficult part is not checking one status—it is reconstructing what happened across provider evidence, webhook delivery, merchant state, and application logic without allowing an AI model to invent payment truth.
PayTrace explores a focused developer workflow for investigating that mismatch with real PayPal Sandbox evidence, deterministic detection, and evidence-grounded AI assistance.
What it does
PayTrace demonstrates one complete incident investigation:
- Create, approve, and Capture a real PayPal Sandbox order.
- Store PayPal state separately from merchant-local order state.
- Record an allowlisted evidence timeline with provenance and verification status.
- Reproduce a clearly labelled merchant-side update failure without changing PayPal evidence.
- Wait for an application-defined grace period before deterministic reconciliation opens an Incident.
- Generate a structured, read-only investigation with GPT-6.1 Terra.
- Validate every AI citation against stored evidence.
- Replay or reset the local merchant scenario without another PayPal Create, Capture, refund, or webhook-redelivery call.
The project deliberately distinguishes real provider evidence from simulated merchant behavior. It does not simulate a PayPal outage and does not give AI authority over financial or merchant state.
How we built it
The frontend is a responsive React and TypeScript application. The backend uses Express and TypeScript with a layered structure of routes, workflow services, provider adapters, shared contracts, and a transactional SQLite repository.
PayPal Orders v2 is used for server-side order creation and Capture. PayPal webhook events are accepted only after official postback signature verification succeeds. Provider event IDs are deduplicated transactionally, and unknown order mappings are quarantined.
The deterministic reconciliation rule compares a confirmed PayPal Capture with merchant-local state after a configurable grace period. Only after an Incident exists does the OpenAI Responses API receive a minimized evidence pack.
GPT-6.1 Terra returns strict structured output containing facts, hypotheses, unknowns, and safe next steps. Zod validation, evidence-ID validation, payment-state boundary checks, and financial-action checks gate every report. Invalid or unavailable AI output falls back to an honest rule-based report.
Challenges we ran into
The most important challenge was preserving trust boundaries. PayPal COMPLETED and merchant PAID are different states and must never be conflated.
Webhook verification also required sending PayPal the complete original event while persisting only allowlisted fields. Another challenge was defining the reconciliation grace period correctly for both new Captures and retained evidence.
Strict structured AI output introduced latency and validation challenges. The application therefore uses a bounded timeout, permits one validated correction attempt, and falls back safely when output remains invalid.
Finally, local Replay had to remain useful without accidentally creating another payment. Tests verify that Replay and Reset do not call PayPal Create or Capture.
Accomplishments that we're proud of
- Completed and persisted genuine PayPal Sandbox Captures.
- Verified a real app-linked PayPal webhook through PayPal's official verification endpoint.
- Kept provider state, merchant state, and simulated evidence visibly separate.
- Implemented deterministic and idempotent Incident creation.
- Generated a live GPT-6.1 Terra investigation with resolvable evidence citations.
- Confirmed that AI investigation does not mutate payment or merchant state.
- Built safe local Replay and Reset controls.
- Passed 35 automated tests, TypeScript lint, and production build verification.
What we learned
Reliable payment investigation depends more on evidence boundaries and state modelling than on generative output. AI is most useful after deterministic systems establish the incident and when every generated claim can be traced back to stored evidence.
We also learned that absence of a webhook is not proof that PayPal did not send one, asynchronous systems need explicit grace periods, and reproducible local fault injection can be more valuable to developers than an unsupported root-cause prediction.
What's next for PayTrace
PayTrace is currently a Sandbox-only developer lab. Future work could include additional merchant-owned failure scenarios, sanitized regression-test fixture export, broader payment-provider support, and production-grade deployment architecture.
The current project intentionally excludes live funds, refunds, automatic remediation, multi-tenant authentication, and production merchant data.
Built With
- api
- express.js
- gpt-6.1
- node.js
- openai
- orders
- paypal
- react
- responses
- sandbox
- sqlite
- structured
- terra
- typescript
- v2
- vite
- vitest
- webhooks
