Inspiration

When money moves through PayPal, not every payment cleanly matches an invoice. Missing references, partial payments, split settlements, overpayments, refunds, and currency mismatches create reconciliation exceptions.

Finance teams often resolve these manually by comparing payment records against invoices and internal ledgers. This is slow and error-prone. More importantly, payment memos are untrusted user-controlled text. A malicious memo could attempt to manipulate an AI-assisted workflow into approving a fraudulent allocation.

We wanted to build an autonomous reconciliation agent that automates the investigation while keeping financial decisions deterministic, independently verified, and under human control.

What it does

The PayPal Reconciliation Agent sits between PayPal and a financial ledger to investigate unmatched payments safely.

Its core capabilities:

  1. PayPal ingestion: Connects to PayPal Sandbox through OAuth 2.0, Invoicing API v2, and Transaction Search API to ingest invoice and payment records.
  2. Deterministic matching: Matches payments against open invoices using evidence-first algorithms. Supports exact matches, partial payments, split allocations, overpayments, refunds, and reversals.
  3. Independent verification: Rechecks proposed allocations against database source records. Amount alone can never justify an allocation.
  4. AI exception investigation: Uses local Qwen3:8b through Ollama to investigate ambiguous cases using read-only database tools and structured evidence citations.
  5. Fail-closed safety verification: The AI has zero ledger mutation permissions. An independent verifier rejects ungrounded claims and hallucinated evidence.
  6. Prompt-injection defense: Treats payment memos and references as data, never instructions. Suspicious content triggers deterministic alerts and locks approval.
  7. Human review: Presents proposals and evidence through a React dashboard. Operators retain control over financial approvals.
  8. Atomic approvals and audit: Uses PostgreSQL transactions and row-level locks to prevent double allocations. Decisions are recorded in a tamper-evident, SHA-256 hash-chained audit trail.

How we built it

We designed the system as a decoupled, multi-layer architecture.

  • Reconciliation core: A framework-independent Python package containing deterministic matching algorithms, financial arithmetic using integer minor units, benchmark generation, and proposal verification.
  • Storage: PostgreSQL, SQLAlchemy 2.0, and Alembic migrations manage financial records and independent payment, reconciliation, and allocation states.
  • Backend: FastAPI provides review queues, investigation endpoints, tenant scoping, and audit verification.
  • AI subsystem: A model-agnostic investigator uses local Qwen3:8b through Ollama and read-only tools to gather evidence.
  • Frontend: React 19, TypeScript, Vite, and Tailwind CSS power a monochrome operator dashboard.
  • PayPal integration: OAuth 2.0, Invoicing API v2, Transaction Search API, and webhook signature verification.

We tested against actual PayPal Sandbox transactions. Invoice RECON-T-901 for $42.50 USD was paid by a Sandbox buyer and reconciled into transaction 89H37763BV8635251.

Challenges we ran into

1. Asynchronous PayPal reporting

Invoice payment information and transaction reporting data did not always become available simultaneously. We built ingestion logic to link records using PayPal transaction IDs and reconcile both sources without duplicate settlements.

2. Grounding the AI investigator

During early experiments, Qwen3:8b failed our independent verifier 60 times out of 60 on a complex currency-mismatch scenario. Instead of weakening the verifier, we improved evidence schemas, structured output formats, and context handling.

Our later evaluation also exposed limitations: Qwen3:8b achieved 50% classification accuracy and 75% evidence validity across four representative cases. One unsupported claim was rejected by the independent verifier and marked INVALID.

3. Preventing financial race conditions

Concurrent approvals can cause over-allocation. We implemented PostgreSQL row-level locks and rechecked payment capacity and invoice balances atomically during approval.

4. Designing for financial clarity

We chose a monochrome interface rather than relying on conventional green and red status badges. Shape, border weight, typography, and explicit locked states communicate the difference between verified facts, advisory warnings, and blocked actions.

Accomplishments we're proud of

  • 1,042 passing backend tests and 36 skipped tests.
  • 33 passing frontend tests.
  • Successful TypeScript checks and production frontend build.
  • Real PayPal Sandbox ingestion and reconciliation.
  • 100% precision and 66.7% recall on the separate deterministic-v1 matcher benchmark.
  • A fail-closed verification architecture that rejects unsupported AI claims.
  • Prompt-injection scenarios that keep financial approval locked.
  • A tamper-evident SHA-256 hash-chained audit trail.

What we learned

An LLM should be an advisor, not an accountant. Financial matching and verification need deterministic rules. AI is useful for explaining ambiguous exceptions, but it should not authorize financial mutations.

We also learned that payment memos must be treated as untrusted input, idempotency is essential in payment pipelines, and independent verification is more valuable than simply making an AI model appear accurate.

What's next

  • Granular multi-tenant role-based access control.
  • Continuous PayPal webhook ingestion.
  • Deterministic multi-currency settlement handling.
  • Better confidence calibration for AI investigations.

The goal is to turn reconciliation from a manual exception-handling process into a verifiable, auditable workflow.

Built With

Share this project:

Updates

Submission history