-
-
PayPal evidence, deterministic rules, AI investigation, and human review.
-
Three deterministic scenarios with PayPal identifiers masked in Judge Mode.
-
A USD 20 expected refund is compared with the verified USD 5 PayPal refund.
-
AI separates facts from uncertainty while humans retain decisions and audit history.
Inspiration
Merchants often store order expectations separately from payment records. A completed order may be unpaid, underpaid, or incompletely refunded, while the reason for the difference remains unknown.
OrderMatch AI was created to turn these payment exceptions into evidence-grounded investigations without allowing AI to invent financial facts or make financial decisions.
What it does
OrderMatch AI connects merchant order expectations with real PayPal Sandbox Orders, Captures, and Refunds.
Deterministic rules identify incomplete payments, amount mismatches, and refund mismatches using integer-cent calculations. Every finding retains its underlying PayPal evidence.
AI receives a structured evidence package and produces four clearly separated sections:
- Finding
- Verified Evidence
- Uncertainty
- Recommended Next Step
Financial findings are rebuilt from deterministic rules, so the model cannot override the calculated result.
Human reviewers can manage investigation status, add notes, inspect evidence changes, and follow an audit timeline. Financial status and human investigation status remain independent.
A server-enforced Judge Mode masks PayPal identifiers, blocks write operations, and reads only cached AI investigations matching the current evidence.
How we built it
The application uses Node.js 24, native HTTP, SQLite, and a framework-free browser frontend.
PayPal Sandbox Orders v2 provides checkout and Capture evidence. PayPal Payments v2 provides refund evidence. The webhook pipeline verifies PayPal signatures before processing events and deduplicates successfully processed event IDs.
Reconciliation is performed by deterministic JavaScript rules using integer cents. DeepSeek receives only a structured package containing the relevant stored evidence.
The hosted demo runs behind Nginx and HTTPS, with Node.js bound to localhost and SQLite used for persistent evidence and investigation history.
Challenges we ran into
The main challenge was keeping financial truth separate from model-generated interpretation.
We built validation and grounding layers that reject invented transaction identifiers, unsupported causes, unsafe financial actions, malformed output, and incomplete provider responses.
Another challenge was providing a useful public demo without exposing payment controls or private transaction identifiers. This led to the server-enforced read-only Judge Mode.
Accomplishments that we're proud of
OrderMatch AI demonstrates three evidence-backed scenarios:
- An incomplete payment with no completed Capture
- A six-dollar payment amount mismatch
- A fifteen-dollar refund mismatch
The application combines real PayPal Sandbox evidence, deterministic reconciliation, structured AI investigation, human review, evidence fingerprinting, and an append-only audit timeline.
The project includes 60 automated tests covering reconciliation, AI grounding, Judge Mode, identifier masking, webhook processing, investigation state transitions, and safe fallback behavior.
What we learned
AI is most trustworthy in financial operations when it investigates evidence rather than deciding financial truth.
Deterministic systems should establish what happened. AI can then explain the finding, identify uncertainty, and help a human decide what to investigate next.
What's next
Next steps include validating sustained external webhook delivery, adding versioned database migrations, expanding route-level integration tests, and supporting additional evidence sources without weakening the deterministic reconciliation boundary.
