Inspiration

Last year, a freelancer friend lost a \$2,400 project payment to a chargeback. He had delivered the work — the client even thanked him over email — but when the dispute hit, he had 10 days to respond and no idea what evidence PayPal needed. He submitted a few screenshots. PayPal ruled against him.

He didn't lose because he was wrong. He lost because he was unprepared.

That story haunted us. We dug into the numbers: merchants lose over \$100 billion annually to payment disputes worldwide. Yet the average merchant spends just 48 minutes preparing a dispute response — against professional chargeback teams who do this full-time. The asymmetry is brutal.

We asked ourselves: What if a merchant never had to scramble? What if every transaction was continuously monitored for dispute risk, and when a dispute did arrive, an AI that had been watching the entire transaction lifecycle could instantly assemble a courtroom-grade defense — grounded in real data, not hallucinated summaries?

That's how ProofRail AI was born: a payment assurance platform that doesn't just react to disputes — it prevents them.

What it does

ProofRail AI is a full-stack payment assurance and dispute defensibility platform built on PayPal's APIs. It operates across three layers:

🛡️ Layer 1 — Financial Guard Rails Before any payment is processed, our RejectMoneyFieldsInBody middleware intercepts every HTTP request and blocks 12 categories of monetary fields (amount, currency, total, value, price, cost, fee, tax, shipping, discount, unit_amount, purchase_units) from client-submitted request bodies. This makes price manipulation structurally impossible — not just validated, but architecturally blocked at the HTTP layer. All monetary values originate exclusively from the server database.

🔍 Layer 2 — Pre-Dispute Health Monitoring Every transaction is continuously scored across 6 risk vectors using Google Gemini AI:

  • Delivery Proof — tracking numbers, carrier data, delivery confirmations
  • Communication Trail — buyer-seller message history and response times
  • Refund History — past dispute patterns and frequency
  • Transaction Velocity — unusual timing or volume anomalies
  • Amount Anomaly — deviation from the buyer's normal purchase patterns
  • Account Age & Trust — verification status and historical behavior

Merchants see a real-time health score for every transaction. When risk spikes, they get actionable alerts before a dispute is filed — time to add tracking, send a follow-up, or issue a preemptive partial refund.

📋 Layer 3 — AI Dispute Dossier Engine When a dispute does arrive (via PayPal webhook), ProofRail's AI engine generates a comprehensive 12-section dispute response dossier:

  1. Executive Summary
  2. Transaction Timeline (with precise timestamps)
  3. Payment Verification
  4. Delivery Evidence
  5. Product/Service Description
  6. Communication Log
  7. Refund Policy Compliance
  8. Prior Transaction History
  9. Risk Assessment (across all 6 vectors)
  10. Evidence Inventory with SHA-256 integrity digests
  11. Recommended Response Strategy
  12. Supporting Documentation

Every claim in the dossier cites specific database records — not generic AI text. Evidence files include cryptographic SHA-256 hashes for tamper-proof integrity. The dossier is then submitted directly to PayPal through the Provide Evidence API (/v1/customer/disputes/{id}/provide-evidence) as a multipart upload.

How we built it

Architecture Philosophy: We built ProofRail as a production-grade distributed system, not a hackathon prototype. Every component was chosen to handle real-world payment workloads.

Backend — Go (Gin + GORM) We chose Go for its concurrency model and low-latency performance — critical when PayPal webhooks require sub-second acknowledgment. The API server runs as two horizontally scaled pods behind Nginx with least-connections load balancing.

Frontend — React 19 + TypeScript + Tailwind CSS The merchant dashboard uses AG Grid for real-time transaction tables, custom risk visualization gauges, and PayPal's JavaScript SDK for live checkout flows. An onboarding wizard guides merchants through PayPal sandbox setup.

AI Engine — Google Gemini (Multi-Model Cascade) We use gemini-2.5-flash as our primary model with temperature 0.1 for near-deterministic financial analysis. But here's what makes our AI pipeline safe:

gemini-2.5-flash → gemini-2.0-flash → gemini-1.5-flash → deterministic heuristic

If all three Gemini models fail (rate limits, network issues, API outages), the system falls back to a keyword-based heuristic engine that uses regex pattern matching to produce risk scores. The system never fails silently — it always produces an assessment. In fintech, a "sorry, AI is unavailable" message is unacceptable.

Event Pipeline — Kafka + Redis + MySQL

  • Kafka (KRaft mode) streams PayPal webhook events through a consumer group for reliable async processing
  • Redis provides Cache-Aside caching (60s TTL on disputes), SETNX-based distributed locks for webhook idempotency (24h TTL), and a write-behind event queue
  • MySQL 8.0 serves as the persistent store with GORM batch inserts
  • Write-Behind Batching: Timeline events are pushed to a Redis FIFO list and flushed to MySQL in batches of 200 items or every 2 seconds — whichever comes first — with graceful shutdown drain

Webhook Pipeline (< 5ms response time):

PayPal Webhook → Nginx → Redis SETNX (dedup) → Kafka Publish → 200 OK
                                                      ↓
                                              Background Consumer → AI Analysis → MySQL

PayPal requires fast webhook acknowledgment. Our pipeline returns 200 OK in under 5 milliseconds. The actual processing happens asynchronously via Kafka consumers.

Infrastructure — Docker + Nginx + Caddy + CI/CD

  • 9 containers orchestrated via Docker Compose (2 API pods, 1 worker, Kafka, Redis, MySQL, Nginx, monitoring UIs)
  • Nginx handles rate limiting (100 req/min API, 30 req/sec webhooks), gzip compression, and security headers
  • Caddy provides automatic HTTPS with Let's Encrypt on the production server
  • Jenkins declarative pipeline automates: Go tests → React build → Docker deploy → smoke tests

PayPal API Integration — 8 endpoints, fully integrated:

API What We Use It For
OAuth2 Token Authenticate all server-to-server calls
Create Order Generate payment orders from verified server-side amounts
Capture Order Capture authorized payments after buyer approval
List Disputes Fetch and cache all merchant disputes
Get Dispute Retrieve detailed dispute information for AI analysis
Provide Evidence Upload AI-generated dossiers as multipart evidence
Accept Claim Allow merchants to accept valid claims through the dashboard
Verify Webhook Signature Cryptographically verify every incoming webhook

Challenges we ran into

1. The Go Toolchain Version War Our go.mod specified Go 1.26, but Docker Hub's golang:1.24-alpine image couldn't build it. The go-redis v9.23 library required Go 1.26+. We solved this by pinning golang:alpine (latest) in the Dockerfile and downgrading go-redis to v9.7.0 — maintaining all functionality while ensuring reproducible builds across environments.

2. Making Distributed Systems Fail Gracefully The hardest engineering challenge wasn't making Kafka and Redis work — it was making them fail gracefully. Every external dependency needed a dial-test on startup with automatic fallback:

  • Redis down → direct database queries (slower but functional)
  • Kafka down → synchronous webhook processing (blocking but reliable)
  • All Gemini models down → deterministic heuristic scoring (simpler but always available)

The health endpoint transparently reports: {"redis_cache_available": false, "kafka_stream_available": false} — no hidden failures.

3. Port Conflicts on Production Deployment Our production server was already running llama.cpp on port 8000 and Nginx on port 80. We made the Nginx port configurable via ${NGINX_PORT:-8000}, removed the hardcoded 443 binding (letting Caddy handle TLS termination), and used Caddy as the edge reverse proxy — solving the puzzle without disrupting existing services.

4. The 5-Millisecond Webhook Challenge PayPal can timeout on slow webhook responses and retry — causing duplicate processing. We needed sub-5ms acknowledgment. The solution: Redis SETNX for instant idempotency checking, immediate Kafka publish, and 200 OK before any actual processing begins. The entire hot path avoids database queries.

5. Preventing AI Hallucination in Financial Documents AI-generated dispute responses can't contain fabricated claims. Our solution: the AI receives only structured data from the database (order records, payment captures, timeline events) and its output is forced into a JSON schema with response_mime_type: "application/json". Every evidence item in the dossier includes the SHA-256 hash of the original record, making any tampering or hallucination detectable.

Accomplishments that we're proud of

🏆 The Financial Guard Middleware — RejectMoneyFieldsInBody is our proudest innovation. While most payment applications validate amounts on the server, they still accept them from the client. We made it structurally impossible for a malicious client to inject monetary values. This isn't validation — it's architectural prevention. We haven't seen this pattern in any other hackathon project or open-source payment system.

🏆 Production-Grade Distributed Architecture in a Hackathon — 9 Docker containers, Kafka event streaming, Redis write-behind batching, multi-pod load balancing, CI/CD pipeline, and automatic HTTPS — all working together. This isn't a demo; it's a system that could handle real PayPal transaction volumes.

🏆 Deep PayPal Disputes API Integration — We didn't just call /create-order. We integrated the entire dispute lifecycle: listing disputes, analyzing them with AI, generating evidence dossiers, and submitting evidence back through the Provide Evidence API with multipart file uploads. Most hackathon projects stop at checkout. We built what happens after checkout goes wrong.

🏆 The AI Safety Floor — Our 4-tier AI fallback (3 Gemini models + deterministic heuristic) ensures that a financial risk assessment is always produced, regardless of API availability. In production fintech, "the AI is down" is never an acceptable answer.

🏆 Source-Grounded, Tamper-Proof AI Output — Every claim in our AI-generated dispute dossiers cites specific database records with SHA-256 integrity digests. This makes ProofRail's output legally defensible — auditors can verify that the AI's claims match the original evidence, byte for byte.

What we learned

Distributed systems are about failure, not success. Building Kafka and Redis integrations was straightforward. Making them fail gracefully — with transparent health reporting, automatic fallback, and no data loss — was 10x harder and 10x more important.

PayPal's Disputes API is powerful but underutilized. The Provide Evidence endpoint accepts multipart uploads with structured evidence. Most merchants don't know it exists. Automating evidence submission through this API is a genuine competitive advantage.

AI in fintech needs guardrails, not just models. Setting temperature to 0.1, forcing JSON schema output, citing source records with cryptographic hashes, and maintaining a heuristic fallback — these aren't optional safety measures. They're the difference between a toy and a tool that handles real money.

The "last mile" of deployment is brutal. Getting 9 containers running locally is one thing. Getting them running on a production server with HTTPS, behind Cloudflare, with existing services occupying standard ports — that's an entirely different engineering challenge that forced us to think about operational reality.

Middleware is the strongest form of validation. You can validate inputs in handlers, services, or models — but middleware that runs before routing makes invalid states unrepresentable. Our financial guard middleware doesn't check if amounts are valid; it ensures they never arrive in the first place.

What's next for ProofRail AI

📊 Real-Time Dispute Analytics Dashboard — Aggregate dispute patterns across all merchants to surface industry-wide fraud trends, seasonal risk spikes, and evidence strategies with the highest win rates.

🔗 Multi-PSP Support — Extend beyond PayPal to Stripe Disputes API, Square Disputes, and Adyen chargebacks — giving merchants a single pane of glass for all payment provider disputes.

📱 Mobile Merchant App — Push notifications when a transaction's health score drops below threshold. One-tap to view the AI-recommended action. No merchant should learn about a dispute from a PayPal email.

🧠 Dispute Outcome Learning — Feed dispute resolution outcomes back into the AI model to improve risk predictions. Transactions that look healthy but result in disputes are the most valuable training signals.

🏛️ Regulatory Compliance Module — Auto-generate evidence packages that meet specific chargeback network requirements (Visa CE 3.0, Mastercard GCMS) with jurisdiction-aware formatting.

⚡ Real-Time Streaming with WebSockets — Replace polling with server-sent events for live dashboard updates when webhook events arrive, disputes are filed, or health scores change.


Built With

Share this project:

Updates

Submission history