Inspiration

Every payment fraud story we kept reading had the same pattern: the payment system worked exactly as designed.

Scammers weren't necessarily hacking PayPal or exploiting vulnerabilities in its infrastructure. Instead, they manipulated users through fake customer-support calls, urgent payment requests, phishing links, and impersonation tactics to convince victims to authorize legitimate payments to fraudulent recipients.

That asymmetry bothered us. Most traditional fraud detection systems focus on analyzing transactions after or during processing, but social engineering begins much earlier — in the conversation surrounding the payment.

We asked ourselves a different question: What if the security layer existed between the user and the "Pay" button?

One principle shaped our entire development process:

Scammers don't need to hack the payment system if they can convince the user to authorize the payment.

GuardPay was built to detect that manipulation before money moves.

What it does

GuardPay is a PayPal-native payment safety layer designed to detect and prevent social-engineering scams before payment authorization.

It protects users against fake support calls, impersonation, urgent payment requests, suspicious invoices, and phishing links.

Key capabilities include:

  • Guarded Checkout: A unified payment flow collects transaction context, including amount, recipient, merchant, message, and URL. The frontend sends context only, never risk scores, verdicts, or security-related boolean flags.
  • Automatic Signal Collection: The backend derives transaction history, recipient reputation, payment velocity, content and URL patterns, network information, blocklist matches, device context, and behavioral signals from trusted sources.
  • Deterministic Risk Engine: A weighted scoring engine evaluates triggered signals and returns one of three decisions: ALLOW, VERIFY, or STOP, accompanied by explainable findings.
  • AI-Powered Fraud Intelligence: Groq-backed AI agents extract structured evidence related to impersonation, urgency, psychological pressure, secrecy, scam patterns, and suspicious URL structures. The backend validates and correlates this evidence with deterministic signals.
  • Controlled Verification: High-risk-but-not-blocked transactions require server-side TOTP or simulated challenge verification before any PayPal order can be created.
  • PayPal Sandbox Integration: Supports recipient-bound PayPal Orders v2 creation, human approval, server-side payment capture, and signature-verified, idempotent webhooks.
  • Android Call-State Detection: A Kotlin-based Android WebView shell detects active phone calls and reports call-state context, helping identify payments initiated during suspicious support calls.
  • Explainable Transaction History: Every risk decision records its findings, AI status, PayPal references, processing latency, and verification outcome. Live security alerts are delivered through Server-Sent Events (SSE).

How we built it

We designed GuardPay around four services, keeping the architecture modular, auditable, and testable.

Technology Stack

  • Backend: Java 17, Spring Boot
  • Frontend: React, Vite, TypeScript
  • AI Service: Python 3.12, FastAPI, Groq
  • Database: PostgreSQL, Flyway
  • Infrastructure: Docker Compose
  • Mobile: Kotlin, Android WebView
  • Payment Integration: PayPal REST APIs (OAuth 2.0, Orders v2, Webhooks)

Architecture

  1. Spring Boot — Trusted Business Layer: Handles authentication, owner-scoped data access, trusted signal collection, deterministic risk evaluation, PayPal orchestration, TOTP verification, security alerts, and SSE streaming. Authentication includes BCrypt password hashing, opaque HttpOnly cookies, CSRF protection, and session rotation and expiry.

  2. React + Vite — User Interface: Provides guarded checkout, transaction verification, AG Grid-powered transaction history, and the security dashboard.

  3. FastAPI — AI Intelligence Layer: Runs two focused LLM agents for extracting structured social-engineering and scam evidence, alongside always-on local structural URL analysis rules. We deliberately avoided CrewAI and other agent frameworks in favor of a fixed, predictable, and testable pipeline.

  4. PostgreSQL — Persistent Data Layer: Stores users, recipients, merchants, transactions, payment context, risk assessments, verification outcomes, and audit information, with Flyway managing database migrations.

Deterministic Risk Scoring

The core risk engine uses a reproducible weighted sum of triggered signals:

$$ S = \min\left(100,\sum_{i \in \text{triggered}} w_i\right) $$

The final action is determined by configurable thresholds:

$$ \text{Action} = \begin{cases} \text{STOP} & S \geq \tau_{\text{stop}} \text{ or blocklisted}\ \text{VERIFY} & \tau_{\text{verify}} \leq S < \tau_{\text{stop}}\ \text{ALLOW} & S < \tau_{\text{verify}} \end{cases} $$

All signal weights are maintained centrally in a single RiskSignal enum, making scoring transparent and auditable rather than dependent on an opaque learned model.

For behavioral baselines, we use rolling 7-, 30-, and 90-day windows with robust statistical measures such as median and Median Absolute Deviation (MAD). This prevents a single unusual transaction from distorting a user's normal spending profile.

AI contributes additional evidence, but the deterministic engine remains the sole authority responsible for the final payment decision.

Challenges we ran into

1. Keeping AI powerless over payment decisions

It was tempting to let the LLM make the final fraud decision. However, LLM outputs can be inconsistent and difficult to reproduce. We designed an AI corroboration gate where uncorroborated AI social-engineering findings remain auditable but unweighted. AI evidence cannot independently change an ALLOW decision into VERIFY or STOP.

2. Ensuring that security signals come from trusted sources

Initially, client-supplied risk hints could potentially be manipulated. We removed client-controlled risk verdicts and security flags entirely. Network information is accepted only from trusted proxies, and missing information is explicitly reported as unavailable rather than incorrectly assumed to be safe.

3. Preventing duplicate payments

Retries, network failures, and accidental double-clicks can result in duplicate payment attempts. We implemented idempotent order creation and capture operations so repeated requests return the previously stored result instead of creating duplicate transactions.

4. Building a reliable evaluation process

We separated our evaluation data into a 66-scenario development set and a 32-scenario PayPal-specific held-out set. We ensured that scenario IDs and inputs remained disjoint, preventing accidental tuning against held-out evaluation data.

Accomplishments that we're proud of

  • Built a complete guarded payment workflow covering risk evaluation, verification, PayPal Sandbox order creation, human approval, payment capture, and transaction history.
  • Implemented a deterministic risk engine that achieved 100% exact score, risk-level, and action reproducibility on our benchmark.
  • Achieved 100% signal precision and recall on labeled benchmark scenarios.
  • Improved AI-assisted fraud detection recall from 87.1% to 93.6% on the development dataset, with F1-score improving from 83.1 to 86.6, without giving AI independent decision-making authority.
  • Integrated real PayPal Sandbox APIs, including OAuth authentication, recipient-bound Orders v2, server-side capture, and verified webhooks.
  • Implemented an Android call-state signal to identify payments initiated during active phone calls.
  • Built a comprehensive automated testing suite with 867 automated tests across the stack, including 480 backend tests, 55 frontend tests, 32 AI service tests, and additional benchmark suites.
  • Created a CI-friendly safety-invariant demonstration script to validate critical security guarantees.

What we learned

  • Explainability is a product feature: Users are more likely to trust a payment being blocked when they can understand exactly why it was flagged.
  • Deterministic systems matter in financial applications: Reproducibility, auditability, and predictable behavior are essential when real money is involved.
  • AI should strengthen evidence, not own decisions: LLMs are valuable for understanding unstructured conversations and identifying manipulation patterns, but payment authorization must remain under deterministic control.
  • The authorization step is critical: Traditional fraud prevention often assumes users make informed decisions. Social engineering exploits precisely the situations where users are manipulated into making unsafe decisions.
  • Data quality matters as much as model quality: Trusted signals, robust behavioral baselines, and careful evidence validation are essential for reliable fraud detection.

What's next for GuardPay

GuardPay currently operates with PayPal Sandbox integration. Our next steps focus on production readiness, broader fraud intelligence, and stronger verification.

  • Production PayPal Integration: Move from Sandbox to live PayPal integration with production-grade device attestation and security controls.
  • Shared Fraud Intelligence Network: Expand cross-user recipient reputation, reporting, review, and fraud-outcome sharing to identify suspicious recipients across users.
  • Adaptive Risk Calibration: Calibrate signal weights and thresholds using real, labeled fraud outcomes while preserving deterministic and explainable decision-making.
  • Advanced URL Intelligence: Integrate live URL reputation services such as Google Safe Browsing to identify malicious and phishing destinations.
  • Stronger Authentication: Introduce passkeys, MFA at login, and additional verification channels.
  • Production-Ready Android Protection: Strengthen the Android application with Play Integrity attestation and reliable background-safe call-state detection.

Our long-term vision is to make payment security proactive rather than reactive — detecting manipulation before a user authorizes a payment, not after the money is gone.

Built With

  • ag-grid
  • android-webview
  • docker-compose
  • fastapi
  • flyway
  • groq
  • hibernate
  • java
  • junit
  • kotlin
  • openai-compatible-api
  • paypal
  • paypal-orders-v2
  • paypal-sandbox
  • paypal-webhooks
  • postgresql
  • python
  • react
  • server-sent-events-(sse)
  • spring-boot
  • tailwind-css
  • totp
  • typescript
  • vite
  • vitest
Share this project:

Updates

Submission history