VerifyBeforePay

AI + Cybersecurity

Try the working browser app · Watch the public functional video (3:19) · Open the MP4 directly · Source and README

Problem and intended use

A message can look familiar while asking for a code or payment through an unrelated link. VerifyBeforePay helps the person receiving that request pause, inspect its wording and actual link host, and choose an independent way to confirm it. It is intended for individuals checking suspicious messages, not as a bank's authentication system.

The sender always remains unverified. A low model score, HTTPS, or a matching domain never becomes a “safe” verdict. The intended benefit is a clearer verification step before disclosure or payment; reduced losses and other user outcomes have not been measured.

Working components

A learned multinomial Naive Bayes model uses word and bigram features, fixed Laplace smoothing of 1, and an elevated-signal threshold of 0.9. It is a small classical ML model, not an external inference API. Separate Python checks identify urgency, credential, payment, and secrecy wording, extract the real URL host, and flag user information or an expected-domain mismatch. Context and negation can change what those wording cues mean.

The static browser app runs the unchanged Python analysis through self-hosted Pyodide and verifies the fixed code and model hashes before use. Only app files are downloaded; message text is not uploaded. Pasted links are not visited, and the app uses no analytics or message storage. Clear removes the input and results from the interface, not securely from memory. A standard-library Python-only loopback version is also included. No API key, paid dependency, or account is needed to test the app.

Evaluation and limits

One preregistered, normalized-duplicate-grouped evaluation used 4,136 training groups and 1,023 untouched test groups from the older English UCI SMS Spam Collection. Spam precision was 0.992, recall 0.908, and F1 0.948; the fixed keyword baseline's F1 was 0.566. The model had 119 true positives, 1 false positive, 12 false negatives, and 891 true negatives. The holdout was evaluated once and was not used for retuning.

Twenty-four authored functional cases checked output behavior. Another 34 synthetic software cases matched complete outputs and validation errors between native Python and the browser runtime. These are not modern fraud benchmarks. Chrome checks covered actual deployed behavior, invalid input, clearing, and a narrow mobile layout.

New scams and unsupported languages can be missed. The app does not check reputation, account state, sender identity, or voice deepfakes. The README states the split, metrics, limitations, and local test commands.

Work completed during ForgeHacks

The app, evaluated model, browser packaging, public deployment, and functional video were created after the October 3 kickoff. Reused components are Python, Pyodide, and the public UCI dataset. Coding assistants helped implement and check the project; that assistance is separate from the project's own learned model. The edited video shows actual public app screens with fictional examples and synthetic eSpeak NG narration. It is not a continuous recording.

Dataset: Almeida & Hidalgo (2011), SMS Spam Collection, DOI 10.24432/C5CC84, CC BY 4.0. No raw SMS rows are bundled. Runtime licenses and attribution are retained in THIRD_PARTY_NOTICES.md.

Built With

• css
• html
• javascript
• naive-bayes
• pyodide
• python
• webassembly