PayShield AI

An AI agent that reads the message behind a payment, in Arabic or English, and stops scams before PayPal sends the money. Caught 92% of scams in 159 real Saudi messages.

Comment

Inspiration

In Saudi Arabia, almost everyone gets messages like these: "Your Saudi Post parcel is on hold, pay 3.50 riyals here", "Your Absher account will be suspended", "I sent you a code by mistake, send it back". They are short, written in Gulf Arabic, and look exactly like the real thing. The scam doesn't happen inside the payment app. It happens in the message that convinces someone to pay.

The scale is real. A 2025 Consumers International survey found that more than half of Saudis have experienced fraud, and only 43% of victims got their money back. In a Kaspersky survey, 58% of people in Saudi Arabia said they had faced scams by text message or phone call.

Payment platforms see the transaction, not the WhatsApp chat or SMS behind it. I wanted to build the step in between: something that reads that message, investigates it like a careful friend would, and decides whether the money should move, in the language people are actually scammed in.

This is personal for me. When I moved to the United States in 2017, I was 18 and my English was still limited. I received scam messages all the time, and every one of them scared me, because I couldn't always tell what was real and what was a trick. That made me think about everyone in the same position: older people, people reading in a language that isn't their own, anyone who is anxious and in a hurry. Scammers count on exactly that. I built PayShield so that nobody has to figure it out alone, and so it explains the danger in the language they understand best.

What it does

PayShield sits in front of PayPal checkout. You paste the message that asked you to pay, or upload a screenshot of it, and PayShield gives one of three decisions:

  • SAFE TO PAY: the PayPal checkout button appears.
  • VERIFY FIRST: you answer a few safety questions. Any "No" turns it into a block.
  • BLOCKED: the payment is refused. The server will not create a PayPal order, even if someone edits the page.

What happens in between:

  1. Quick check (under a second). A rule engine reads Arabic, English and Arabizi (Arabic typed in Latin letters and numbers, like ad5ol 3la elrabt for "click the link"), undoes tricks like P@yPal or split letters, and highlights the suspicious words in the original message.
  2. AI investigation. An AI agent decides which tools to use: it checks each link, expands short links to see where they really go, and looks up how old the website is. Every step it takes is shown on screen in plain English or Arabic.
  3. Final score. The agent's verdict decides the outcome, with a short explanation in both English and Arabic. Payment stays locked until it finishes.

It also handles details that matter for Gulf users. Amounts can be entered in Saudi riyals or UAE dirhams: PayPal doesn't process those currencies, so PayShield charges the exact USD amount at the official fixed rate. And if the AI is unavailable, the rule engine still protects the payment.

How it fits with PayPal: PayPal's own scam alerts look at the payment. PayShield looks at the message behind it, the SMS or chat that made someone want to pay, and works for any payment type.

How I built it

  • Backend: Python and Flask. Every check is stored on the server, and the PayPal endpoints refuse to create or capture an order unless that check allows it (HTTP 403 for blocked, pending or unverified checks).
  • PayPal: Orders v2 API (OAuth client credentials, create and capture) and the PayPal JavaScript SDK buttons, in the sandbox.
  • AI agent: Gemini with function calling. The model chooses tools in a loop, and the tools run in parallel to keep it fast: a link checker, a short-link expander (guarded against requests to internal addresses), domain age through RDAP, Google Safe Browsing when a key is set, and the rule engine itself. Screenshots are read by the same model.
  • Rule engine: patterns for Arabic, English and Arabizi scams, link analysis (shorteners, risky domain endings, raw IP links, lookalike domains by edit distance), and a normalizer that undoes disguises while keeping a map back to the original text, so highlights land on the right characters.
  • Two-stage result: the rules answer instantly, the agent runs in the background, and the page updates when the final score arrives.
  • Frontend: plain HTML, CSS and JavaScript, fully bilingual with right-to-left Arabic.
  • Hosting on Render: the live demo runs as a Render web service defined in a Render Blueprint (render.yaml): one-click deploy, auto-deploy on every push, a health check, and per-visitor and daily AI limits so the free AI quota can't be used up.
  • Evaluation: a benchmark script, a dataset builder that turns screenshots into a labelled CSV, and automated tests with a mocked AI agent.

How well it works

I collected 159 real messages from Saudi Arabia, with private details covered: 64 scams (fake delivery fees, bank and government impersonation, account takeover, fake savings circles, fake tickets, marketplace scams) and 95 genuine messages (bank alerts, one-time codes, delivery updates, marketing, personal chats). Most are in Arabic, with English and mixed-language messages too.

Version Scams caught Scams blocked outright Real messages flagged Accuracy
v1 87.5% 14.1% 15.8% 85.5%
v2 (final) 92.2% 82.8% 15.8% 87.4%

To keep the numbers honest, the 159 real messages were only used as a final exam. Every improvement was built and checked on a separate set of 137 generated messages that I reviewed by hand, and the real set was run once at the end.

The trade-off is real: v2 blocks more real marketing messages that look like scams (10 of 95, up from 5), such as prize ads and recharge offers. Most of the remaining missed scams are fake event tickets and fake government notices with no link to investigate.

Challenges I ran into

  • No public dataset of Gulf scam messages. I built one from screenshots: an AI model transcribes each image, conversations split across several screenshots are merged, and duplicates are removed. Then I checked every row by hand.
  • Real security messages look like scams. A genuine bank code says "do not share this code", which a keyword rule reads as credential theft. The fix was to tell the difference between a message that gives you a code and one that asks you to send it.
  • Combining rules and AI. My first version averaged the two scores, so a confident AI scam verdict got watered down by a low rule score, and a single rule could block a message the AI had checked and found safe. Now the agent's investigation decides, and rules alone can only ask the user to verify.
  • Speed. Early agent runs took over 40 seconds or timed out. Choosing a faster model, lowering its thinking level, running tools in parallel and returning the rule result instantly brought it down to a few seconds.
  • Free-tier limits. The benchmark hit Gemini's per-minute and per-day limits, so it saves every answer and resumes where it stopped.
  • No riyals on PayPal. PayPal doesn't process SAR, so PayShield converts at the official fixed rate of 3.75 per dollar.

Accomplishments I'm proud of

  • It doesn't just warn. A blocked check cannot become a PayPal order, and that is enforced on the server.
  • It works in the language people are actually scammed in: Gulf Arabic, formal Arabic, English and Arabizi.
  • It was measured on real messages, with a method that keeps the numbers honest, and v2 blocks 83% of real scams outright, up from 14%.

What I learned

  • A benchmark only means something if you never tune on it. Keeping a separate check set changed how I worked.
  • False alarms matter as much as catches. A tool that blocks your real bank codes gets turned off.
  • An AI agent is most useful when it gathers evidence with tools, not when it just reads the text.

What's next

  • Fewer false alarms on marketing: teach the agent the official domains and sender names of more Gulf brands.
  • Scams without links: fake event tickets and government notices need different evidence, such as checking whether an event or a service really exists.
  • Meet people where the scam arrives: a browser extension that checks a page before the PayPal button is clicked, and forwarding a suspicious WhatsApp or SMS to get a verdict back.
  • More real data, collected with consent, to retrain and re-measure.

Built With

Share this project:

Updates

Submission history