Sentin AI

AI-powered autonomous PayPal dispute resolution — retrieves evidence via RAG, computes optimal strategies with EV math, and executes actions via PayPal APIs

Comment

Inspiration

Online merchants lose billions annually to chargebacks. Each PayPal dispute triggers a non-refundable $15–$20 network fee, hours of manual evidence assembly, and escalating merchant risk scores. We asked: what if an AI system could autonomously defend merchants — retrieving proof-of-delivery, computing whether to fight or settle using real math, and executing the optimal action via PayPal's APIs in seconds?

What It Does

Sentin AI is an autonomous risk operations command center for e-commerce merchants. It:

  1. Ingests disputes in real-time via PayPal webhooks or a 1-click scenario injector for live demos.
  2. Retrieves contextual evidence — store return policies, carrier Proof-of-Delivery (PoD) records with signature confirmation, and buyer dispute history — using Elasticsearch 8.x Hybrid RAG (BM25 keyword matching + dense vector kNN cosine similarity).
  3. Evaluates claims with AI — Google Gemini 2.0 Flash with native Pydantic structured outputs reasons over the evidence to determine win probability and recommended action.
  4. Calculates optimal strategy using Expected Value (EV) economics:

$$\text{EV}(\text{Contest}) = P(\text{win}) \cdot \text{Amount} - (1 - P(\text{win})) \cdot (\text{Amount} + \text{Fee}) - \text{Labor}$$

  1. Autonomously executes low-risk, high-confidence decisions (evidence submission via PayPal /provide-evidence or refunds via /v2/payments/captures/{id}/refund), while routing borderline cases to a human supervisor.
  2. Renders an executive command matrix in AG Studio / AG Grid with filterable columns, AI confidence pills, and slide-out inspection drawers showing full evidence trails and EV breakdowns.

How We Built It

  • Backend: Python 3.11 / FastAPI with async SQLAlchemy 2.0, Pydantic v2 strict schemas, and Alembic migrations on PostgreSQL 16.
  • AI Engine: Google Gemini 2.0 Flash via the google-genai SDK with response_schema=AIDecision for guaranteed structured JSON outputs. A deterministic rule-based fallback engine ensures 100% demo uptime even without API credentials.
  • Search & RAG: Elasticsearch 8.x with hybrid BM25 + dense vector cosine similarity indices over merchant policies, carrier tracking logs, and buyer profiles. A zero-config NumPy in-memory fallback provides seamless offline operation.
  • PayPal Integration: Full PayPal REST API integration — OAuth2 token caching (/v1/oauth2/token), dispute evidence submission (/v1/customer/disputes/{id}/provide-evidence with PROOF_OF_FULFILLMENT), claim acceptance, and payment capture refunds (/v2/payments/captures/{id}/refund) with idempotency keys.
  • Frontend: Next.js 14 (App Router) + React 18 with AG Studio / AG Grid React for the interactive risk matrix, Tailwind CSS for utility-first styling, and Lucide icons.
  • Infrastructure: Docker Compose for local development (PostgreSQL 16 + Elasticsearch 8.15), Render Blueprint for cloud deployment.

Challenges We Faced

  • PayPal Sandbox API quirks: The Disputes v1 sandbox endpoints have limited test dispute availability. We built a robust simulation engine (routes_simulate.py) with 6 scenario types (INR with/without tracking, SNAD minor/major, fraud patterns, high-value) that create realistic dispute records with full AI triage pipelines.
  • Ensuring 100% demo reliability: External APIs (PayPal, Gemini, Elasticsearch) can be intermittent. We engineered graceful fallbacks at every layer — mock tokens for PayPal, deterministic heuristics for the AI engine, and NumPy vector search for RAG — so the demo never crashes regardless of connectivity.
  • EV math precision: Getting the Expected Value formula to correctly model contest vs. settle economics with dispute fees, labor costs, and variable win probabilities required careful derivation and break-even probability analysis.

What We Learned

  • Hybrid search (BM25 + dense vectors) dramatically outperforms pure keyword or pure semantic search for retrieval-augmented generation over structured business documents.
  • Gemini's native Pydantic schema enforcement (response_schema) eliminates brittle JSON parsing and guarantees type-safe AI outputs.
  • Human-in-the-loop guardrails are essential — autonomous execution should only trigger when confidence is high, amounts are below a threshold, and the buyer isn't flagged for abuse patterns.

What's Next

  • Real-time webhook ingestion from live PayPal merchant accounts.
  • Multi-seller tenant isolation with per-seller PayPal credential vaults.
  • Advanced analytics dashboards with dispute trend forecasting and chargeback rate prediction.

Built With

Share this project:

Updates

Submission history