The Good Scammer

Inspiration

I'm a 30-year-old software engineer, and I almost fell for a scam call.

The caller knew my full name, my phone number, where I live, and that I had an Oney card. None of that was secret. Details like these leak in data breaches all the time, from any website or shop that has ever stored them, and scammers can buy them.

I build software for a living, and I still almost fell for it. I was shaken, and one question stayed with me: if scammers are this convincing with someone like me, what chance do our parents and grandparents have?

The numbers aren't reassuring. In Spain, fraud made up about 89% of the 464,801 cybercrimes recorded in 2024. In the US, the FBI reports that people over 60 lost $7.7 billion to online crime in 2025, about 60% more than the year before. $352 million of that was lost to AI-powered scams, including cloned voices of family members "in distress."

Warnings and leaflets tell people that scams exist, but they don't let anyone practise. We built The Good Scammer so that the first convincing scam call your grandmother gets comes from us, not from a real scammer.

What it does

The Good Scammer is a scam-training platform for elderly people, set up by the family members who look out for them.

A relative signs up and adds the phone number of a parent or grandparent. The elderly person installs nothing.

A welcome call explains that practice calls are coming and asks for their agreement.
Our AI places a realistic mock scam call, starting with a fake bank fraud alert asking them to "verify" their card.
We evaluate the call, flagging what was revealed (card number, PIN, one-time codes) without storing the details themselves.
The call ends with a gentle reveal and one simple tip, so nobody is left scared or confused.
The relative gets a notification and a short brief: what was shared, which red flags were missed, and how to start the conversation.

How we built it

Contract-first, so two halves could be built at once. The API contract lives in backend/openapi.json, exported from the FastAPI code; orval generates the TypeScript client, types and mock handlers from it. Nobody hand-writes an API type, and CI fails if either half drifts. That let the React app be built against generated mocks while the backend was still stubs.

Backend: Python 3.12, async FastAPI in strict layers — views → services → repos → models, with adapters/ as the only place that ever sees a provider credential. SQLAlchemy 2.0 async on Postgres 17, Alembic migrations, Pydantic settings that refuse to boot half-configured rather than failing at the first call.

Frontend: React 19, TypeScript strict, Vite, TanStack Query for all server state, Tailwind v4 with a single set of semantic tokens so light and dark come from one palette. LiveKit drives the in-browser practice call.

Voice: four SLNG agents — practice and consent, each in Spanish and English — published from version-controlled prompt files, never edited by hand in a dashboard. They dial out over a Twilio Elastic SIP trunk. Twilio also sends the sign-in codes, because sign-in is a phone number and a six-digit SMS code and nothing else: an account has no email address at all.

Deployed to Cloud Run with Cloud SQL, through GitLab CI using Workload Identity Federation — no service-account key exists anywhere.

Challenges we ran into

Consent is the entire product, and it was unrecordable. A voice provider will tell you a call rang, was answered, and ended. It will not tell you what the person said. We discovered this the hard way: one of us deliberately said "no" on a real call, and the system recorded nothing — then invited us to ring them again. The fix was a shared tool the agent invokes mid-call to report the answer, keyed to a correlation id we mint ourselves, because the model cannot reliably know the provider's internal call id.

Two words nearly barred a consenting person forever. Our Spanish/English classifier read "no problem" as a decline — it contains "no", and a decline is permanent and attaches to the number. Someone agreeing cheerfully would have been locked out with no way back. Separately, apostrophe folding turned "I don't know" into i don t know, so no hedge matched and someone who simply hadn't followed the question read as refusing.

A staging flag would have given away every account. ENVIRONMENT=staging isn't "production", so the fixed development sign-in code stayed active. The moment we made the URL public, anyone could have typed any phone number plus a known six digits and been signed in as that person. We caught it before opening the door, and tied the two settings together so they can only change as a pair.

The platform's write format wasn't its read format. We inferred the voice-agent config shape from the API's GET response. Four consecutive rejections taught us otherwise — a read-only field, an empty object failing a discriminated union, a tool schema missing a parameter, a variable that had to be declared in two places. Each cost a failed publish.

A trailing newline cost an afternoon. openssl rand -hex 32 appends one. The secret was 65 bytes, the token ended in \n, and every authenticated request failed with a flat 401 — while both values looked identical in any console.

Accomplishments that we're proud of

The system refuses to invent consent. There is one state machine, one writer, and active is reachable from exactly one event: an answer we actually heard. A test walks the transition table and fails if that stops being true. When a call ends and nobody can say what happened, the status is answer_unknown — a real state, not a guess in either direction.

"We couldn't hear the answer" is a feature. It would have been easy to infer agreement from a two-minute call. We'd rather show a relative an honest blank than a confident lie about whether their mother said yes.

Transcripts are read and dropped. The answer is classified in memory; nothing is stored, logged or echoed back. We store the category of what was disclosed — card number, PIN, one-time code — never the value. The call stops at the first sign of a real secret.

The UI never fabricates. Sentences like "she gave nothing away on her last three calls" are only written when the data exists. Where it doesn't, the page says so. Invented results on a page about a real person's safety are worse than an empty section.

No impersonation. "Your bank's security department" — never a real bank's name, never a spoofed number.

What we learned

The hard part isn't making the AI convincing. It's making it stop. Building a believable scam call took a prompt. Making it reveal itself, stop at the first real secret, honour a spoken "stop", and never ring someone who hasn't agreed — that was the whole engineering effort.

A safety rule that isn't recorded isn't a safety rule. Our screen told relatives a spoken "stop" was "honoured straight away". The caller did stop. Nothing wrote it down, so the scheduler would have drawn the next call. Telling someone we'd stopped and then ringing their father anyway is the worst thing this product could do — and it read as true until we traced it end to end.

Guard the thing you can't see. Every expensive bug came from trusting a model of a system instead of checking it: the vendor's write format, our own test environment, a secret's byte length. The tests we value most now pin what we verified, with the error they prevent named in the test.

Build the refusals first. Consent, the safety stop, the "we don't know" state — these are the product, not the guardrails around it.

What's next for The Good Scammer

Listen to a full call, end to end. Everything is wired and tested; what no one has yet confirmed is that the agent performs the safety stop and the reveal with a real person on the line. That's the gate before anyone outside the team hears it.
Honour a spoken "stop" automatically — the agent says it, the system must record it.
Evaluation and the email brief: red flags missed, what was disclosed, how a family can follow up — currently the largest unbuilt piece.
Run the scheduler so practice calls actually arrive in their windows, unannounced.
Account recovery. Sign-in is a phone number; lose the SIM and the account goes with it.
More scenarios beyond the bank fraud alert — delivery fees, the grandchild in trouble, the cloned voice.
More languages, once English and Spanish have been heard in anger.