CaptureProof

A successful API call isn't proof of payment. CaptureProof uncovers PayPal checkout mismatches, links AI explanations to source code, and verifies developer repairs against recorded evidence.

Comment

Inspiration

A checkout can look successful even when the payment isn't complete.

While working with PayPal integrations, I became interested in a problem that ordinary API testing can easily miss: the payment provider and the merchant application can disagree about whether money has actually been captured.

A request might return successfully while the capture is still pending. Meanwhile, the merchant application could already have marked the payment as completed. Finding the cause means piecing together provider responses, database records, execution history, and source code.

I built CaptureProof to make that investigation easier. I wanted developers to see not just that a payment integration failed, but where it failed, why it happened, and whether a repair actually changed the outcome.

What it does

CaptureProof is an AI-assisted debugging tool that connects PayPal payment evidence, merchant database states, source code, and repair verification.

Rather than relying on API success messages, it examines what the payment provider reported and what the merchant application actually recorded.

The included example demonstrates a concrete integration failure: a capture remains PENDING at the provider, yet the original implementation records completed payment and creates a capture row in its database.

CaptureProof turns this into an investigation developers can follow:

  • Overview: Compare the original failure with a separately tested developer repair at the same execution checkpoint.
  • Evidence: Follow recorded payment-state transitions and inspect the exact observations behind each result.
  • AI review: Read a real saved OpenAI explanation that connects the failure to specific source-code conditions, with clickable citations.
  • Repair: Inspect the implementation changes and compare their recorded outcomes against the original behavior.
  • Files: Download the evidence, frozen source code, and replay artifacts. The self-contained HTML report also works offline.

A separate executable application lets developers work with supported integration candidates and review test plans before execution.

The main idea is simple: a diagnosis should lead to evidence you can inspect, code you can trace, and a result you can verify.

How we built it

I built CaptureProof using Node.js and JavaScript, integrating Medusa 2.7.0, PostgreSQL, and the PayPal Server SDK 1.0.0. OpenAI provides source-grounded diagnostic explanations.

The featured case runs real Medusa services, database operations, and SDK integration code with controlled, simulated PayPal responses. This makes it possible to reproduce specific payment-state transitions and examine what the application actually persists.

The verification layer uses deterministic rules to identify mismatches between provider observations and merchant records. The AI layer works with the captured evidence and source code to explain why those mismatches occurred.

In the featured execution, the saved OpenAI analysis identifies a specific problem: retrievePayment accepts an order with result.id without confirming capture completion. The host then sets captured_at using its locally calculated isFullyCaptured value instead of confirmed provider capture status.

CaptureProof links that explanation directly to the relevant source lines, allowing developers to inspect the code behind the diagnosis.

I also implemented and tested a separate developer repair that checks completed payment evidence and preserves the information needed to reconcile later capture results.

During development, I additionally tested genuine PayPal Sandbox checkout and response-discard recovery scenarios, separately from the featured simulated-provider example.

The final reporting interface brings the execution evidence, AI explanation, source references, and repair observations into one navigable, portable report.

Challenges we ran into

One of the hardest parts was handling payment states correctly across multiple layers.

A provider capture can be pending when the first request returns and complete later. The merchant application needs to recognize those transitions without recording completion too early or losing track of an existing capture during recovery.

That meant examining more than response codes. I had to trace execution order, inspect persisted database states, distinguish repeated requests from distinct captures, and compare original and repaired behavior under matching conditions.

Another challenge was making the evidence understandable without losing the technical detail. A payment trace can contain a large amount of information, but a developer still needs to find the important state change, understand the cause, and reach the relevant code quickly.

I worked through that by organizing the report around the investigation itself, keeping detailed evidence and source files available through citations and expandable sections.

Accomplishments that we're proud of

The most important result is a reproducible payment integration bug with a separately implemented and tested repair.

At the first checkpoint, both implementations observe a PENDING provider capture. The original incorrectly records a completed merchant payment and one active capture row. The repair keeps the local state authorized and records zero capture rows.

At a later checkpoint, the provider capture becomes COMPLETED. The original merchant record still shows PENDING, while the repair correctly records COMPLETED.

These are concrete differences in observed application behavior, not just AI-generated recommendations.

Other accomplishments include:

  • Connecting a real saved OpenAI diagnosis to exact source-code references and recorded execution evidence.
  • Comparing original and repaired behavior across multiple recorded checkpoints.
  • Preserving source files, replay results, and supporting artifacts in a portable HTML report.
  • Building an interface that makes detailed payment evidence accessible without requiring developers to read raw logs first.
  • Completing 234 related regression checks and 207 isolated-distribution checks, with some overlapping coverage.

The result is a working debugging workflow where developers can move from a payment-state mismatch to its cause, inspect the repair, and review the evidence supporting the outcome.

What we learned

The biggest lesson was that a successful API call and a completed payment are not the same thing.

Reliable payment integrations need to account for state transitions, persisted records, retries, and the evidence required to establish that a capture actually completed.

I also learned that AI is considerably more useful for debugging when its reasoning is connected to something developers can verify. An explanation becomes much more actionable when it points to the exact condition in the source code and the execution evidence behind it.

Building CaptureProof reinforced the importance of combining deterministic checks with AI-assisted investigation, rather than asking AI to be the final authority on payment correctness.

What's next for CaptureProof

I want to expand CaptureProof to more PayPal integration workflows and additional payment failure scenarios, building on the same evidence-driven approach.

The next priorities are making setup easier for other developers, improving integration with existing testing workflows, and expanding the scenarios that can be investigated and compared.

Longer term, I want CaptureProof to help developers catch subtle payment regressions earlier and make payment integration debugging more reproducible.

The goal is to make it easier to answer three questions that matter whenever payment code changes: What went wrong? Why did it happen? Did the fix actually work?

Built With

Share this project:

Updates

Submission history