Inspiration
Online payments make commerce easier, but they create real problems for merchants. Fraudulent transactions cause direct financial losses, and wrongly rejecting legitimate customers quietly costs sales. Most fraud tools stop at producing a score and leave the merchant to work out what to do with it.
We wanted to address the whole problem, not just the model. Our goal was a platform that helps merchants assess payment risk, understand why a transaction looks suspicious, make informed decisions, and run payment operations reliably. That became ReturnGuard, an AI-powered payment risk intelligence platform.
How we built it
ReturnGuard is an independent platform that communicates with merchant applications through APIs and a native React SDK.
- Backend: Python, FastAPI, and SQLAlchemy. Risk scoring uses LightGBM and XGBoost.
- Payments: PayPal integration for authorization, capture, and void, with webhook verification, event deduplication, and reconciliation.
- Reliability: explicit payment lifecycle states, idempotency keys, a transactional outbox, retry handling, and reconciliation for uncertain outcomes.
- Investigations: a reviewer state machine (open, in review, approved, closed, reopened) with an append-only step log. An LLM drafts case narratives as an advisory output only, and a human must approve them before anything is submitted.
- Frontend: a React SDK and dashboard, using AG Grid Community for transaction and review tables.
Our first merchant integration, ShopPoint, is an e-commerce app. Its backend handles orders, authentication, and payment execution, while ReturnGuard supplies the risk intelligence that informs each payment decision. The platform is multi-tenant, so many merchants share infrastructure while their data stays isolated.
Key formulas
Risk-based decision. Each transaction gets a fraud probability ( p \in [0, 1] ) from the model ensemble, and the policy maps it to an outcome:
$$ \text{action}(p) = \begin{cases} \text{allow} & p < \tau_{\text{review}} \ \text{manual review} & \tau_{\text{review}} \le p < \tau_{\text{block}} \ \text{block} & p \ge \tau_{\text{block}} \end{cases} $$
Rate limiting on admin login. The sliding window allows at most 5 attempts per IP in any 60-second window. For earlier attempt times ( t_i ) from one IP, a request at time ( t ) is accepted only if
$$ \left|{\, t_i : t - 60 < t_i \le t \,}\right| < 5 $$
Idempotent payment retries. A payment request carries an idempotency key ( k ). Repeating the request has the same effect as running it once:
$$ f(f(x; k); k) = f(x; k) $$
Challenges we ran into
Connecting predictions to real payments. A risk score only matters if it can influence a payment safely and consistently. We had to define explicit states and policy rules so the model's output became a deterministic, auditable action.
Uncertain outcomes. A timeout does not mean a payment failed. The provider may have processed it while our application never got the response. We handled this with idempotency keys, explicit payment states, and reconciliation against the provider.
Reliable async processing. Payment and review events must survive service failures. We built a transactional outbox with retries, leases, and duplicate-event handling.
Multi-tenant security. Merchants must only see their own transactions, API operations, and investigations. We enforced tenant scoping at the data layer and tested it directly, including cross-tenant access returning 404.
Explainable decisions. A bare number doesn't help a reviewer act. We surfaced the top risk contributors, deterministic findings with source references, and an evidence completeness checklist.
Accomplishments that we're proud of
- A passing test suite: 449 Python tests (9 skipped), 26 SDK tests, and 6 React component tests, with no regressions across phases.
- An investigation workflow where every action lands in an immutable timeline, and approval is invalidated automatically if the evidence pack changes.
- A risk-path invariance test showing the LLM layer has no effect on model scores.
- A secure admin flow with one-time API key reveal, hashed keys, and HTTP-only, Secure, SameSite=Strict session cookies, plus an automated check that no secrets are bundled into client assets.
What we learned
Payment risk management involves much more than machine-learning accuracy. We learned to design distributed workflows around failure, use idempotency to make retries safe, keep risk intelligence separate from payment execution, and maintain consistency across asynchronous systems.
We also learned that a prediction alone doesn't make a payment system trustworthy. A reliable platform must explain its decisions, enforce security boundaries, handle failures gracefully, and give merchants tools to act on what it tells them.
What's next for ReturnGuard
We want ReturnGuard to become reusable payment risk infrastructure that merchants can integrate without building their own risk systems. Next priorities:
- Expanding merchant integrations beyond ShopPoint
- Improving developer onboarding and API-key management
- Strengthening risk models and explainability
- Extending refund processing and reconciliation
- Building deeper operational analytics
Built With
- ag-grid
- artificial-intelligence
- backend
- commerce
- fastapi
- fraud-detection
- lightgbm
- machine-learning
- multi-tenancy
- payment-processing
- paypal
- python
- react
- rest-api
- typecsript
- typescript
- xgboost