Unhook

Shows you the exact words a scam message is using to manipulate you, and what to do next. Built for older adults.

Comment

Inspiration

Older adults lose more money to fraud than any other age group, and every tool built to help them answers the same question: is this a scam, yes or no?

That's the wrong question. A verdict protects one message and teaches nothing — so the next scam, arriving by a route the filter never sees, works just as well. And these messages genuinely don't arrive as spam. They come as a text from a real phone number, a WhatsApp from someone who seems to know you. The person is the filter.

The question a frightened person actually has is "why shouldn't I trust this?" Unhook answers that one: it marks the exact words doing the manipulating, names the technique, and gives one instruction.

What it does

Paste a suspicious message. Unhook returns a plain-words verdict, the message with the manipulative phrases highlighted, an explanation of each, one thing to do right now, and a projection of the scam's next three moves. It reads aloud, and it answers in whatever language the message was written in.

How we built it

Five layers, in TypeScript on Next.js:

  1. Normalise — strip zero-width characters, NFKC, catch Cyrillic homographs
  2. Deterministic detectors (~2ms, free) — 16 signal types: lookalike domains, crypto addresses, gift-card requests
  3. The model, in two passes — one asks only for evidence against a 15-tactic taxonomy; the second does only the writing
  4. Span resolver — converts quotes to character offsets
  5. Risk rubric — published constants

Three decisions shaped everything.

The model never returns a score. It returns evidence; a rubric you can read and argue with turns that into a number. Self-reported LLM confidence is uncalibrated and shifts when you reword the prompt.

The model never returns character offsets. Models quote reliably and count characters unreliably. So it returns the verbatim quote and a three-tier resolver computes offsets where it can be unit-tested — measured at 1.000 match quality across 166 findings.

The two detection layers are blind to each other, which makes their disagreement informative. The UI surfaces it rather than hiding it.

Challenges we ran into

The false-positive problem is the whole problem. Put a red banner on someone's real bank text and they learn your warnings are noise. So we built 27 of our 38 genuine test messages to look like fraud — real 2FA codes, a real bank fraud alert, a hospital saying call urgently.

Measuring instead of guessing. Six experiments over 81 hand-labelled messages, scored with \( F_1 = 2 \cdot \frac{P \cdot R}{P + R} \).

Pattern checks alone, versus pattern checks plus the language model:

  • recall: 18.6% → 97.7%
  • false positives: 0.0% → 0.0%
  • F1: 0.314 → 0.988
  • hard-case recall: 9.1% → 100%

Assumptions that were wrong. Gemini's free tier turned out to be 20 requests/day, not 1,500 — it died mid-run. Haiku 4.5 rejects the thinking parameters Opus accepts. A cosmetic 200-character cap silently threw away three valid analyses and biased the sample, because it only fired when the model was wordy.

A cache that failed silently. We built the production bundle before generating the demo cache, so an empty cache was baked in and every example quietly fell through to a live API call. Nothing errored. There's now a test for it.

What we learned

Prompt engineering buys precision, not recall. Strip the taxonomy out and recall rises while the same model raises seven false alarms where it had raised none — including that Barclays alert at 89/100.

Splitting the job helps. Asking one call to find evidence and write for a frightened 74-year-old does both worse. Two passes moved the action line from reading grade 9.3 to 4.5, for +18% cost.

Measure the reading level. Flesch–Kincaid grade:

$$0.39\left(\frac{\text{words}}{\text{sentences}}\right) + 11.8\left(\frac{\text{syllables}}{\text{words}}\right) - 15.59$$

Our hand-written copy averaged grade 5.8; rewriting took it to 2.9. The lever was sentence length, not word length.

The cheapest model wasn't the cheapest. Opus 5's 512-token cache minimum clears our prompt (87.8% hit rate); Haiku's 4096-token minimum never engages. The highest list-price model was the cheapest arm to run.

Calibration, reported honestly. ECE 0.145, Brier 0.072, monotonic — but systematically under-confident. That's the safe direction for a tool whose worst failure is crying wolf, and we report it rather than tuning the thresholds to flatter the chart.

What's next

Screenshot support, since most people receive scams as images. A wider labelled set collected with fraud-prevention charities rather than written from published patterns. And per-tactic scores for the two techniques our current set under-measures.

Built With

Share this project:

Updates

Submission history