Inspiration
Small online sellers often have to manage payment disputes on their own. They must understand the claim, collect order and delivery information, prepare evidence, and respond before the deadline. This process can take time and can be difficult without a dedicated support team.
We wanted to make this work easier with an AI assistant. At the same time, we did not want an AI system to make financial decisions or submit dispute responses without the seller's knowledge.
This led us to build PayPal AI Dispute Copilot: an assistant that prepares information and recommendations while keeping the merchant in control.
What it does
PayPal AI Dispute Copilot is an AI-assisted dashboard that helps merchants review and manage PayPal disputes from one place.
- Dispute Management: View, filter, sort, and review disputes in a central dashboard.
- Deadline Monitoring: See upcoming response deadlines and identify disputes that need attention first.
- Evidence Preparation: Gather available order, delivery, and tracking information and prepare a structured response.
- Historical Case Search: Search similar synthetic past-dispute examples with Elastic Cloud to add useful context.
- AI Drafting: Generate and revise dispute evidence with Groq AI. Merchants can add notes and request a new draft.
- Win Probability Estimate: Show an estimate based on available evidence and similar cases. It is an estimate, not a guarantee of the outcome.
- Human Approval: Require the merchant to review and approve a response before the application submits it.
- Audit History: Record drafts, merchant edits, and approval events in a chained SHA-256 audit ledger.
- Dispute Simulation: Simulate incoming disputes to test and demonstrate the workflow.
How we built it
We built the application with a React 18 and Vite frontend and a Python FastAPI backend.
The frontend uses AG Grid to display and manage disputes. It also includes a dispute detail panel, a deadline timeline, an audit trail view, and a dispute simulator.
The backend manages dispute data, API routes, PayPal synchronization, webhook events, evidence drafts, approval actions, and audit records. We use a shared data contract so that the frontend, backend, and AI services work with the same dispute structure.
For AI evidence drafting, we integrated Groq using the openai/gpt-oss-120b model. The agent uses available dispute details, supporting information, historical examples, and merchant notes to prepare a response. We added prompt guardrails to reduce unsupported claims and to instruct the model not to invent tracking details or documents.
For historical search, we integrated Elastic Cloud and indexed 30 synthetic past-dispute examples. We also added an in-code fallback so the application can still provide search context when the external search service is unavailable.
For delivery evidence, we integrated KERNEL to obtain carrier tracking and delivery information for supported carriers, including UPS, FedEx, USPS, and DHL.
For PayPal connectivity, we used the PayPal Sandbox REST API. The integration includes OAuth 2.0 client-credentials authentication, dispute synchronization, evidence submission, and webhook signature verification. We also added webhook event deduplication.
A key part of the design is the human approval gate. The backend controls the approval state. The AI prepares a draft, but it cannot independently approve or submit it. The merchant must review and authorize the action.
Challenges we ran into
One challenge was reducing AI-generated errors. A dispute response must use real supporting information. An invented tracking event or document could harm a merchant's case. We addressed this with structured inputs, evidence-focused prompts, and guardrails against unsupported claims.
Another challenge was separating AI recommendations from actions that affect a merchant's account. We needed to make sure that an AI-generated draft could not bypass the merchant. We addressed this with a backend-enforced approval workflow.
We also had to connect several services with different APIs, authentication methods, and response formats. We separated these integrations into individual backend services and used a shared dispute data contract to keep the application consistent.
Finally, external services may fail or become unavailable. We added fallback behavior for historical search and mock data for development and demonstrations. This helped us test the main workflow without depending on every external service at all times.
Accomplishments that we're proud of
We are proud that we brought several parts of dispute management into one working application: dispute review, deadline tracking, evidence drafting, historical case search, carrier information, and audit history.
We are especially proud of the human-in-the-loop design. The approval requirement is enforced by the backend, not only by a frontend button or instruction to the AI. This reflects our goal of keeping the merchant in control of important financial actions.
We also built a chained SHA-256 audit ledger, a dispute simulator, fallback behavior for search, and automated backend tests. The repository's README reports 12 passing unit tests for the documented test command.
What we learned
We learned that building a useful AI application involves more than connecting a language model to a user interface. The system also needs reliable data, clear service boundaries, input validation, error handling, security controls, and tests.
During this project, we gained experience with:
- Building an AI agent with structured inputs and controlled outputs.
- Integrating REST APIs and handling OAuth-based authentication.
- Using Elastic Cloud to search historical examples.
- Connecting carrier tracking services to an evidence workflow.
- Building a full-stack application with React, Vite, Python, and FastAPI.
- Designing human-in-the-loop approval controls.
- Recording application events with a chained SHA-256 hash structure.
- Testing API routes, agent behavior, and approval workflows.
- Adding fallback behavior for external service failures.
The most important lesson was that trust and human control must be part of the design from the start, especially when AI supports decisions related to money.
What's next for paypal copilot
We plan to improve PayPal AI Dispute Copilot in several areas:
- Expand the historical dataset with more diverse and carefully reviewed dispute examples.
- Improve evidence retrieval and matching between a new dispute and relevant past cases.
- Improve the transparency and calibration of win probability estimates.
- Add more carrier and fulfillment service integrations.
- Strengthen merchant access controls and authorization options.
- Expand automated tests for webhook processing, approval workflows, and external service failures.
- Improve dashboard analytics so merchants can identify common dispute reasons and patterns.
Our goal is to make dispute preparation faster and easier while keeping every important decision under the merchant's control.
