PayResolve AI

Inspiration

We wanted to create a solution that would make our lives easier by solving an issue many people face when making payments online. When processing a payment through a payment gateway, it sometimes shows up as pending or failed on the store.

This can happen when a customer pays through PayPal but the webhook fails to tell the application that an order has been paid for. The business owner then has to investigate the issue across their various systems. This could mean charging the customer again, which could result in them paying twice. If the records aren't cross-checked, it can cause more issues.

We created a targeted system to gather all these details, show exactly why there was a problem with the payment, and help those operating the system take the correct course of action.

This led us to create PayResolve AI, where we combined rule-based detection with AI investigation to make the management of payment exceptions more transparent and logical.

What it does

PayResolve AI monitors your payments and detects any discrepancies between PayPal and your store.

It detects 5 types of anomaly:

Order has been paid but not marked as paid
Amount mismatch
Duplicate webhook events
Refund mismatch
Orphan captures

Features include:

Evidence Graph: Shows the relationships between linked payment, order, webhook, and delivery records. Empty records are shown to highlight where a step has been missed.
AI Investigator: Uses read-only tools to investigate cases. Collects evidence and populates a report with confirmed facts, conflicts, and missing information.
Evidence Validation: Validates record IDs and amounts in AI reports to check they match the database.
Human-Approved Actions: Suggests an action to fix the issue (e.g. update order status). Users with the correct permissions can approve or reject items. All actions are recorded in the audit log.
Demo Shop: Test the effects of webhook failure, duplicate events, dropped webhooks, and webhook handler failure using the Webhook Tester.
Operations and Benchmarking Dashboards: Interactive charts, case metrics, and benchmark results to help users fully understand the system.

How we built it

Frontend

Built with React 19, TypeScript, Vite, and Tailwind v4. We used AG Grid for interactive data tables, AG Charts for operational metrics, and AG Studio to create a customizable operations dashboard. Relationships between evidence are displayed as graphs using React Flow and dagre.

Backend

Java 21 and Spring Boot 4 handle authentication, payment gateway integration, webhook validation, exception handling, the investigation process, and action approvals. PostgreSQL stores payment, webhook, case, and audit records, and schema migrations are handled by Flyway.

PayPal Developer Sandbox was used to create orders, capture payments, validate webhook signatures, and get payment details.

Five rule-based detectors identify rule violations that indicate an exception. When an exception is detected, the backend creates a new case and evidence graph.

AI Investigation

For the AI investigation, we integrated Groq and Google Gemini using Spring AI. Text-based questions go to Groq; questions that include images or PDFs go to Gemini. The AI agent uses read-only tools to collect evidence, and a validation layer makes sure the answers pass checks against the database.

API, Testing and Deployment

We used APIMatic to manage our OpenAPI contracts, and Postman and Newman for API testing. Render is used for deployment and Supabase for hosting our PostgreSQL database.

Challenges we ran into

Payment vs. order state: The biggest challenge was modeling the difference between a successful payment and a successfully updated order. Payments happen in stages, and a webhook can fail, be delayed, or be duplicated.
Webhook reliability: We had to make sure duplicate events were not processed. All events must be stored before processing, and webhook signatures must be validated.
Reliable AI investigation: An AI model can give a plausible explanation that contains missing record IDs or incorrect amounts. To overcome this, we made the investigator read-only and added a validation layer.
Visualizing the evidence: Showing the relationship between the records we had and the ones we didn't know about was an interesting challenge. It helps the operator understand where the exception came from.
Security and engineering practices: We added authentication, RBAC, CSRF protection, audit logging, API testing, and frontend/backend communication through an API contract.

Accomplishments that we're proud of

We are proud that we created a full payment investigation system, not just an AI chatbot or dashboard.

Detecting 5 rule-based payment exceptions
Integrating with PayPal Sandbox to create orders, capture payments, and validate webhook signatures
Creating an AI investigator that queries evidence using read-only tools
Adding a validation layer to confirm AI results match database records
Building an interactive evidence graph with relationships between records and information on missing records
Logging all human-approved corrective actions in an append-only audit log
Creating a sample shop with configurable failure scenarios
Building an operations dashboard using AG Studio, AG Grid, and AG Charts
Adding an OpenAPI-based API contract using APIMatic
Automating tests with Postman/Newman: 21 requests and 132 assertions in the offline collection
Configuring CI workflows and deploying infrastructure

We are proud to have built a system that connects payment detection, investigation, evidence validation, and human approvals.

What we learned

While building PayResolve, we learned how payments work, how they can be affected by webhooks, and the issues that arise when multiple accounting systems store their own transaction records.

We learned how to use PayPal APIs, validate webhook signatures, avoid duplicate event processing, and reconcile payments.

On the AI side, we discovered that simply connecting an LLM to a tool does not create a powerful AI agent. Making permissions, provenance, and validation transparent is just as important.

We also learned best practices for Spring Security, APIs, OpenAPI, databases, frontend/backend communication, and automated testing.

Using AG Grid, AG Charts, and AG Studio showed us how to explore data and how interactive dashboards can be used in real-life situations.

What's next for PayResolve AI

Our next milestone is to move PayResolve AI from our current sandbox environment to a production-ready application.

We would like to:

Support more payment gateways
Improve webhook monitoring and failure handling
Add more payment exceptions and investigation rules
Expand evidence analysis and investigation
Expand benchmarks with more failure scenarios
Add configurable workflows for different companies and operators
Test in production with improved monitoring, scalability, and security
Enhance the operations dashboard with more flexible reports and analytics

We are also interested in learning how PayResolve AI can be integrated into current e-commerce solutions and payment facilitators. This would allow businesses to query their payments without having to completely rebuild their payment system.

We want it to be easier to identify, investigate, respond to, and resolve a payment exception, while still keeping control of the decision.