Inspiration
Online checkout has a trust problem that card networks don't solve. Listing photos are increasingly AI-generated, counterfeits ship from look-alike merchants, and the buyer finds out after paying. We wanted a layer that prices that risk before the buyer pays, lets them opt into protection for a small fee, and settles a valid claim without a support ticket.
The constraint we cared about most was that no model gets to touch money. A single general LLM judging a listing is slow, hard to reproduce, and easy to prompt-inject. So we split the problem: small specialized models produce bounded scores ahead of time, and plain deterministic math turns those scores into a price.
What it does
PayPal Shield scores each listing on five pillars: AI-generated image detection, CLIP image-to-text alignment, review authenticity, a robust Z-score on price, and a Bayesian merchant trust score. A deterministic engine combines them into a risk score and a capped Buyer Protection Fee. The fee is quoted with an HMAC-SHA256 signature and added as its own line item on a PayPal Orders v2 order. If the buyer later files a counterfeit claim with a photo, a claim arbiter decides it, and approved claims trigger an idempotent refund through the Captures Refund API. The verified outcome feeds back into the merchant's reputation, so the next buyer sees a higher price or a block.
The core math fits on one screen:
$$R_{base} = 0.25\,S_{visual} + 0.20\,S_{semantic} + 0.15\,S_{reviews} + 0.15\,S_{price} + 0.25\,S_{merchant}$$
$$R_{final} = \min\big(100,\; R_{base} \cdot M_{penalty} \cdot 100\big), \qquad M_{penalty} \in {1.0,\ 1.5,\ 2.0}$$
$$\text{Fee} = \max\Big(\$0.25,\; \min\big(0.04P,\; P \cdot \tfrac{R_{final}}{100} \cdot 0.05\big)\Big)$$
Merchant trust uses a Bayesian prior so a new account can't look clean by default:
$$\text{Trust} = \frac{\text{successful} + m\,C}{\text{total} + m}, \qquad m = 10,\; C = 0.70$$
Above $R_{final} = 80$, no fee is quoted. Three scenarios, worked by hand: a trusted merchant (190 of 200 orders) selling a $120 item scores 7.80% and pays $0.47. A brand-new merchant at 2 of 2 orders pushing an AI-generated $80 listing scores 70.25% and pays $2.81. A repeat offender with two verified fraud claims hits 81.00% after the 2.0x multiplier and is blocked.
How we built it
It's a modular monolith: FastAPI, PostgreSQL in deployment and SQLite locally, and a React 18 + TypeScript + Tailwind frontend with AG Grid for the risk and reconciliation ledger. We rejected microservices and a message broker because four people and one repo don't need the extra failure surface.
Scores are computed when a listing is created or an item enters the cart, then cached by the SHA-256 of the image plus a hash of the normalized title and description. Checkout only reads the cache. On a miss, the engine uses a conservative 0.50 per uncached pillar and flags the quote MISS_DEFAULT. The merchant pillar is always read live from the ledger, since it's a cheap query and changes with every verified claim.
All money runs in decimal.Decimal with ROUND_HALF_UP. Model outputs enter only through a validator that rejects NaN and infinity and clamps to [0, 1]. A static check fails the build if the engine package contains a float( call. Ownership is strict: each of us has exclusive write access to a directory tree, and only the DevOps and integration owner can change the Pydantic, TypeScript and OpenAPI contracts.
Challenges we ran into
The first was ITEM_TOTAL_MISMATCH. The fee is a second item on the order, so the total must equal the sum of $\text{unit_amount} \times \text{quantity}$ exactly, and JPY, HUF and TWD take no decimals. We enforce equality server-side in Decimal before every call and test it against 1,000 randomized carts.
The second was idempotency under double-clicks and retries. Every PayPal write carries a PayPal-Request-Id derived from a stable internal ID. Refunds are keyed on claim_id with a unique constraint as a second defense, and webhooks are deduplicated on event_id. The reputation increment runs in one transaction with SELECT ... FOR UPDATE on the merchant row, so a claim counts once.
The third was keeping checkout fast when models fail. CLIP doesn't fit comfortably on a 512 MB instance, so it sits behind an API-offload adapter with the same interface. Any provider failure degrades to default scores and a risk-aware quote, never a 500 at checkout.
The fourth was the arbiter. A model can't be allowed to approve a payout, so auto-refund requires four deterministic gates at once: discrepancy above threshold, amount under the automated cap, buyer velocity within limits, and no active dispute. Gate evaluation is a pure function with a full truth-table test. Anything ambiguous goes to manual review.
Accomplishments that we're proud of
Nothing in the money path depends on a model's judgment. The AI produces five clamped numbers, and everything after that is auditable arithmetic that returns identical output for identical input. We back that with 10,000 randomized property tests and the no-float static check.
The closed loop works end to end: a claim goes from photo upload to a real Sandbox refund ID, the merchant's penalty multiplier rises, and the next quote reflects it immediately because trust is derived on read. We also built a deterministic demo mode that serves seeded scores with no network or model calls, so the three scenarios reproduce exactly every time. And four people working in strictly separated directories merged against frozen contracts without stepping on each other.
What we learned
Moving the model out of the request path did more for reliability than any model choice. Treating every AI output as untrusted input to a deterministic engine also removed most of the prompt-injection surface, because no instruction channel is left to inject into. A closed loop only holds up if the write path is boring: unique constraints and row locks did more for us than any retry logic.
We also learned to be precise about what this is. The fee is a platform service fee, not regulated insurance, and we kept that wording consistent everywhere, enforced with a lint check on user-facing strings.
What's next for PayPal-Shield
The biggest gap is the risk pool. A commercial version needs a licensed underwriter holding it, and the real platform_fees split through PayPal partner onboarding in place of our internal ledger simulation. After that comes multi-currency pricing with FX handled properly, shipping and tax line items in protected orders, and chargeback representment so a dispute and a Shield claim don't fight each other.
On the model side, we want to calibrate the claim thresholds and the Z-score constants on real merchant data instead of seeded fixtures, and test the optional vision-language adapter for material and logo checks, with the rule that it can only lower confidence, never approve a payout.
One note before you paste: the Accomplishments section assumes the Sandbox refund flow and tests ran as the spec describes. Cut or soften anything that isn't working by submission.
Built With
- ag-grid
- clip
- docker
- fastapi
- github-actions
- paypal-captures-refund-api
- paypal-orders-v2-api
- paypal-webhooks
- postgresql
- pydantic-v2
- pytest
- python-3.11
- react-18
- render
- sentence-transformers
- sqlalchemy
- sqlite
- tailwind-css
- typescript