Qayğısız

Scam SMS detection that warns the family, not the victim — 90.3% caught on a held-out half of 4,538 real Azerbaijani messages.

Comment

Inspiration

Older people are the ones scam SMS actually works on. They are either frightened by everything or they believe everything, and either way they will not stop to inspect a domain name.

Every app I looked at warns the person who received the message. That is the one person the scam has already fooled. But a warning from their own son or daughter, they believe immediately.

Qayğısız is Azerbaijani for being without worry, without sadness, without fear. That is the state I wanted the phone's owner to stay in.

What it does

Qayğısız runs on the phone of the person being protected. When an SMS arrives, it scores the message on the device itself. If the score crosses a threshold, it sends a Telegram message to a family member, in Azerbaijani, Russian or English, saying what the message claims, which signals fired, and what not to do.

The phone's owner is never asked to decide anything, and sees no alert at all unless they switch one on.

Try it yourself

There are two APKs on the Releases page.

qaygisiz-1.0.3-demo.apk installs in one tap. Open it, paste any message you like into the box, press Check. It gives you the verdict, the score, and every rule that fired, by the same rule codes MEASUREMENT.md reports on. It takes about thirty seconds, and after it you do not have to take a single number in this write-up on trust.

Nothing in this is in your language, so here are four messages to paste. They are taken from the test suite, where their verdicts are pinned, so you can check the code against what you see on screen.

A fake bank link — expect Dangerous Hesabiniz bloklandi. Tecili olaraq http://kapitalbank-az.xyz/giris linkine daxil olun. The host looks like a bank and is not one. Reason code: BRAND_MISMATCH.

A scam with no link at all — expect Dangerous Siz 750AZN mükafat qazanmısınız. Mükafatı almaq üçün 0501234567 nömrəsinə zəng edin! A prize and a number to ring. Four out of five scams this detector used to miss looked like this. Reason codes: PRIZE_CLAIM, CALLBACK.

A real telecom message that uses bit.ly — expect Safe Hormetli Abunechi, Avtomatik balans melumati xidmeti yenilendi. Etrafli: https://bit.ly/balans-az Azerbaijani operators really do send link-shortened SMS. Scoring shorteners as fraud is what caused most of my false alarms, and the corpus is why I stopped.

The one-character scam — expect Dangerous for both spellings Hesabiniz bloklanib, tacili tasdiqlayin Hesabiniz bloklanib, tecili tesdiqleyin Attackers write təsdiq as tasdiq. My normaliser folded ə to e, so the attacker's spelling never matched and a whole campaign walked through. Both fold to the same rule now. Reason code: URGENCY.

That build cannot read SMS and cannot warn anyone, and its home screen says so. It is the full app with the RECEIVE_SMS permission and the two SMS receivers removed by one 25-line manifest file, app/src/demo/AndroidManifest.xml, which you can read in the repository. Nothing else differs. Both APKs are built from the same commit by the same Gradle build and signed with the same key, and the screen you are typing into calls LinkScanner.scan(), the same object the real app calls when a message actually arrives and the same one the 19 tests exercise.

qaygisiz-1.0.3.apk is the real app, and Google Play Protect will refuse to install it. The next section explains why.

How I built it

Android, Kotlin, Jetpack Compose. A BroadcastReceiver reads the incoming SMS and hands it to a rule-based scorer: weighted signals for IP-address URLs, punycode, a mismatch between the link host and the institution named in the body, urgency words, prize claims, premium-rate short codes, and callback numbers in messages with no link at all.

All text is folded before matching, with diacritics collapsed and ə/e/a merged, so təsdiq, tesdiq and tasdiq all hit the same rule.

Detection is entirely on-device. No model file, no network call, no cost per message. A language model is used for exactly one thing: writing the sentence the family member reads at seven in the morning. That is the job a model is actually better at than a rule.

Challenges I ran into

The hardest technical part was keeping API keys out of the repository. The repo is public, and a key in public git history cannot be taken back. I ended up with a security check that runs before every commit.

The one I did not see coming arrived at the very end. I built the app, signed it, published the APK, and then downloaded it onto my own phone, and Google Play Protect refused to install it. Not a warning with an "install anyway" button underneath: a dialog that blocks the install and offers exactly one button, "Got it". Play Protect hard-blocks sideloaded apps that ask for SMS permissions, and reading incoming SMS is the entire purpose of this one. Google's own policy does list anti-SMS phishing as a permitted use of RECEIVE_SMS, but only as a temporary exception, and only for developers who can show a track record of significant protection for users, as reflected in analyst reports, benchmark test results or industry publications. MEASUREMENT.md is the second kind of evidence. It is not a track record, because the app has no users yet. So the one thing this README asks of a reader, do not take my numbers on trust and check for yourself, had become impossible to do. That is why there are two APKs now, and why the difference between them is one small file anyone can read.

Two weeks after publishing, I scanned my own release APK. The commit gate and GitHub's secret scanning both watch the repository, and both reported it clean. But the APK never passes through git. Android Studio builds it and it goes straight to Releases, so neither check could see it. Unpacking it, I found two credentials compiled into the dex: a Telegram bot token and an OpenRouter API key. They were in the real build and in the demo, because both are built from the same Secrets.kt. The token happened to be dead already, for an unrelated reason. The API key was live. I revoked it.

The check I describe above was not wrong, it was incomplete. Keeping a key out of the repository is not the same as keeping it out of what people download, and I had no check on the second thing. There is now a workflow that runs when a release is published. It downloads every asset, unpacks the APKs, and fails if anything inside matches a credential pattern. It prints twelve characters of whatever it finds and never the whole value, because the logs of a public repository are public as well.

The harder part was honesty. "It works" is not a result. I scored the detector against 4,538 real Azerbaijani SMS from a peer-reviewed dataset (Shahbazov 2026, doi:10.25045/jpit.v17.i1.04), split in half with a fixed seed, tuned the rules on one half only, and scored once on the other.

The first number was 19.8% recall. I was sure something was broken in my harness. Nothing was broken. The detector was.

Accomplishments that I'm proud of

After four fixes, on the held-out half:

  • smishing caught: 19.8% → 90.3%
  • legitimate messages falsely flagged: 1.9% → 1.9%, unchanged

Two classifiers trained on the same split reached 84.6% and 81.9% recall at lower precision, and they need a model file. This does not.

The split indices are committed to the repo, so anyone can rerun the measurement and get the same number.

Beyond the number: something that can keep a person safe came out from under my own hands.

What I learned

Measuring showed me three things I had wrong.

  1. Link shorteners are not a scam signal in Azerbaijan. Azercell, Bakcell and mygov send real SMS through bit.ly: 36 legitimate messages against 9 scams. I was scoring that as fraud.
  2. Four out of five missed scams had no link at all. "Ring this number." "Text this word to this short code." I had built a link scanner and called it a scam detector.
  3. One character was costing me a whole campaign. Attackers write təsdiq as tasdiq. My normaliser turned ə into e, so the word never matched.

None of this was visible from reading the code. It only appeared after running it against real data.

What these numbers do not say

Part of the dataset is translated from older English spam. The detector has not yet run against a month of one real person's traffic. 90.3% on this data is not a promise about next month. I also went back and read every message the detector still misses on the held-out half. Twenty-nine messages, each one scoring below the alarm threshold, and ten of them carrying a phone number the dataset itself has redacted, which no rule can score. The same reading shows that one signal, urgency wording, causes eleven of the twenty-nine false alarms on its own, and that the worst false alarm scores higher than every scam the detector missed. That list is published in MEASUREMENT.md, before the version that fixes it exists.

What the app does not do

It does not stop the phone's owner from tapping the link. Android gives no way to block that without becoming the default SMS app, which would make Qayğısız far more invasive than it needs to be. What it relies on instead is speed: the warning reaches the family member within seconds, usually before the owner has decided what to do. That is a real gap, and I would rather state it than let someone find it.

What it would take to publish this

Play Protect blocking the sideload is not only an inconvenience for a reader; it is the distribution problem. An app that reads SMS has to come from the Play Store. That route is a Play Console account, which costs US$25 once and requires the holder to be 18; then a closed test with 12 testers enrolled for 14 consecutive days; then a Permissions Declaration Form for the SMS permission; and only then production access. I am 14, so an adult has to open the account, and the fee is not money I have set aside. What separates this from a published app is administrative, not technical.

The same block is why the full app has never run on a physical phone. What has been verified is the whole chain, on the Android emulator: a message arriving, the receiver firing, the score being computed, and the warning landing in a real Telegram chat from a real bot. What has not been verified is the same chain on real hardware over a carrier network, and I do not have a SIM card either. Closing that gap and reaching a user are the same task.

What's next for Qayğısız

Four of the link rules are language-independent and already work anywhere: IP-address URLs, punycode, a mismatch between the link host and the institution named in the body, and premium short codes. Only the keyword list is Azerbaijani. Adding a country means adding a word list and measuring against that country's data, not rewriting the detector.

  • Collecting real Azerbaijani scam SMS, so that no part of the corpus is translated from older English spam.
  • Russian keyword rules. Azerbaijan is bilingual and the detector currently reads only one of its languages.
  • WhatsApp, which is where much of this has moved. That needs the notification listener, which is on the same Play Protect blocklist as the SMS permissions, so it inherits the same distribution problem.
  • Link and file scanning for the messages that do carry a URL.
  • A shared bot with one-tap pairing, so setup does not require creating a Telegram bot.

Notes

Source, both signed APKs, 19 tests and the exact dev/test split: https://github.com/oomer583/qaygisiz

The narration in the demo video is text-to-speech. My microphone is broken and I do not speak English well enough to record it myself. The script is mine. Music: "Seasons" by Telecasted, from the YouTube Audio Library.

I am 14, in 9th grade in Baku, and I worked on this alone.

On AI: I used AI assistance for code review, boilerplate and the video narration. The idea, the design decisions and the measurement method are mine. I do not write English well, so I answered every question in my own language and used AI to put the answers into English. The content here is mine; the English wording is not.

Built With

Share this project:

Updates

Submission history