Inspiration

Convincing messages can combine urgent requests with a familiar-looking hostname that belongs to an unrelated domain. SignalCheck explores a private, inspectable second look: learned language evidence and actual domain boundaries without opening links or sending messages to a hosted model.

What it does

SignalCheck inspects English message text in the browser. A small trained classifier reports suspicious or ordinary language patterns and abstains when fewer than three tokens are familiar. Independent URL parsing highlights destination-domain mismatch, embedded credentials, IP addresses, HTTP links and internationalized hostnames. It recommends independent verification for sensitive requests. It cannot authenticate a sender and never labels a message safe. The interface supports synthetic examples, custom input, clearing and minimized evidence export. Editing input invalidates the previous report. Runtime inference is local; no links are visited and no messages are stored or transmitted by the application.

How we built it

Created October 7 during ForgeHacks and continued in the same original project; no earlier submitted entry reused. JavaScript vectorizes bag-of-words features; ml-naivebayes trains a real Multinomial Naive Bayes classifier from 40 author-created synthetic English messages. Independent tldts parsing checks hostname boundaries including private suffixes. Lucide supplies icons and esbuild bundles the app. A loopback Node server serves fixed resources with route, Host and CSP restrictions. Coding and documentation assistance: Codex. Project ML is the trained classifier, not a paid API or simulated integration.

Challenges

The small bag-of-words classifier does not reliably understand negation. The benign sentence "The bank will never ask you to share a verification code" is a false positive shown in the demo. Domain matching must reject familiar names embedded inside unrelated domains. Testing found and fixed stale report/export state, overflowing indicator bars, malformed expected hostnames, and URL credential/query words leaking into language indicators.

Verified results

The candidate is locally runnable with a prebuilt bundle, real learned predictions, independent hostname checks and 13 passing automated tests. On 16 distinct same-author synthetic holdout messages: 15 correct, 8 true positives, 7 true negatives, 1 false positive, 0 false negatives. Keyword baseline: 12 correct. Both datasets were written by the same author; this is not a real-world benchmark or general-accuracy claim. The 3:10.24 demo shows five actual captured app states with synthetic narration, including abstention and the known false positive; it is not continuous screen recording.

Engineering lessons

Advisory evidence needs explicit uncertainty and must separate language classification from hostname checks. An ordinary prediction or matching hostname is not proof of authenticity. Export minimization must cover indirect URL-derived word leaks as well as raw text. Clear limitations matter alongside passing tests.

Next steps and limitations

Future work would include independently sourced and responsibly handled evaluation data, broader language/negation coverage and verified mobile/export-download testing. These are plans, not implemented integrations or validated outcomes. Browser download transport and true mobile viewport verification remain unconfirmed. No real-user testing, production deployment or real-world fraud-detection accuracy is claimed.

Run it locally

Public source and README: https://github.com/m7mdd77/signalcheck-forgehacks The README contains Node.js 22+ local-run instructions. No hosted deployment is claimed.