DisputePilot: Evidence-First AI Copilot for PayPal Disputes
The Problem
Payment disputes are difficult for small merchants to manage. When a customer files a dispute, merchants must understand the claim, collect relevant transaction and fulfillment records, identify missing evidence, and prepare an appropriate response within the available timeframe.
These tasks often involve fragmented information and complex payment workflows. More importantly, choosing an action based only on an AI-generated recommendation can be risky: the recommendation may rely on unsupported evidence, overlook case requirements, or suggest an action that is unavailable in the current dispute state.
We built DisputePilot to address this problem by combining AI-assisted reasoning with evidence grounding, deterministic validation, and explicit human approval.
What DisputePilot Does
DisputePilot is an AI-assisted dispute management workspace for PayPal merchants.
It helps merchants:
- Understand disputes: Review dispute reasons, case status, transaction context, and relevant information.
- Ground analysis in evidence: Organize supporting evidence, connect analysis to available evidence references, and identify missing or unsupported information.
- Generate actionable recommendations: Use an LLM to prepare structured dispute analyses and response suggestions.
- Validate actions: Check case state, required evidence, available API actions, and action parameters before execution.
- Maintain human control: Require explicit merchant approval of the exact proposed payload before an action can proceed.
- Track execution: Separate approval from execution and maintain an auditable history of proposals, approvals, execution attempts, and verification results.
The workflow is designed around three important principles: evidence before recommendations, validation before execution, and human approval before consequential actions.
How We Built It
DisputePilot uses a modular architecture:
- Merchant Interface — React, TypeScript and Vite: Provides a workspace for reviewing disputes, examining AI analyses, managing proposed actions, approving operations, and inspecting activity history.
- Backend — Python and FastAPI: Coordinates dispute data, evidence references, AI analysis, validation, approvals, and execution.
- Data Layer — SQLAlchemy and SQLite: Persists cases, analyses, proposals, approvals, execution records, and audit history.
- AI Layer — Local Ollama: Supports optional real LLM inference through a locally configured model. The model produces structured analysis, while application code independently validates evidence references and safety-related fields.
- PayPal Integration — PayPal REST API and Sandbox: Provides OAuth-based authentication and adapters for retrieving dispute information and interacting with supported dispute operations.
The architecture separates data access from AI inference. It also separates analysis, proposal creation, validation, approval, and execution into distinct stages.
Safety and Human Approval
A central design decision is that AI recommendations must never directly authorize financial actions.
DisputePilot applies deterministic validation to check whether an action is permitted by the current case state, whether required evidence is available, and whether the proposed parameters are consistent.
The merchant must explicitly approve the exact action payload. PayPal write operations are additionally protected by configuration and per-action authorization controls.
This design reduces the risk of unsupported recommendations, duplicate submissions, and unintended financial operations.
PayPal and AI Integration
The project includes PayPal Sandbox adapters and a separately configurable local AI provider.
During development, we successfully completed a real PayPal Sandbox seller evidence submission: the provide-evidence request returned HTTP 200, followed by two independent GET requests that verified the submitted evidence and resulting case state.
We also validated local Ollama inference through the FastAPI backend.
These are separate integration tests, not claims that every action is available or that a dispute was won.
Demonstration and Reproducibility
The public release includes an offline demonstration with five synthetic dispute cases.
It demonstrates:
- Reviewing and analyzing multiple dispute cases.
- Preparing and approving a response message.
- Blocking an action when required evidence is missing.
- Waiting when an action is not currently available.
- Preparing and approving a simulated USD 10 partial-refund proposal against a synthetic payment capture.
- Inspecting execution history and audit records.
The default demonstration uses recorded AI analyses and deterministic planning to provide a reproducible experience. It does not make live PayPal requests, submit real evidence, send real messages, move funds, or resolve disputes.
Optional Sandbox integration and fresh local AI inference require the user's own authorized credentials and locally installed model.
Challenges and What We Learned
The most challenging part was not generating text with an LLM. It was ensuring that an AI recommendation remained consistent with the actual evidence, current dispute state, and available API capabilities.
We learned that reliable AI-assisted financial workflows require more than model accuracy. They need explicit evidence references, deterministic validation, human authorization, controlled execution, and independent verification.
DisputePilot demonstrates how AI can assist merchants without taking control away from them.
Evidence first. Human approval. Auditable execution.
Built With
- fastapi
- llm
- ollama
- openai-api
- paypal-rest-api
- paypal-sandbox
- python
- react
- sqlalchemy
- sqlite
- typescript
- vite
