Scout: live scam call protection

Inspiration

Most phone scams don't steal money through a hacked system. They talk a person into sending it. The FBI's 2025 report puts losses for people over 60 at $7.7 billion, much of it moved during a single phone call: a "fraud department" asking to move savings to a safe account, a "grandson" who needs bail, a prize that only needs a small fee. By the time a credit union sees the transfer, the call is over and the money is gone.

We asked a simple question: what if the warning arrived during the call, before the member acts? Scout is our answer, built for Sound Credit Union members.

What it does

When a Sound CU member turns on Scout, calls to their number are forwarded through Scout. Scout listens live, and when the caller asks for money, a code, a password or remote access, it puts the caller on hold and privately tells the member why. The caller hears hold music and nothing else. The member then chooses: continue, end the call, or connect safely to Sound CU.

• Live scam detection in English and Spanish, judged on what the caller asks for, not on keywords
• A neural vocoder artifact screener that flags AI-cloned voices and makes Scout act sooner when a scam is also underway
• An employee portal where Sound CU staff watch flagged calls live (transcript, scam heat gauge, AI-voice badge) and can hold or end the call or freeze the member's outgoing transfers for 24 hours
• Text-message scam filtering for messages from unknown senders

How we built it

• iPhone app (Swift): rings for protected calls on Apple's call screen through Twilio's Voice SDK; hold choices work from the keypad or notification buttons.
• Telephony: Twilio. Each call runs in a conference room, so Scout can hold only the caller and speak to only the member.
• Call engine (Python): live audio from both sides goes to our transcription api for streaming transcription (. About 75 English and Spanish pattern rules fire instantly, and Claude re-checks the conversation after every phrase the caller says.)
• Decision rule: plain code, not the AI, decides when to act. With scam confidence $c$ and an AI-voice flag $v$:

$$ \text{hold} \iff c \ge 0.85 \;\lor\; \big(v = \text{AI} \,\land\, c \ge 0.60\big) $$

An AI voice alone never holds a call, since pharmacy robocalls and appointment reminders use synthetic voices too.

• Website and portal: Next.js on Vercel, Supabase for data and live updates, styled to match soundcu.com.

On live test calls, the time from a scammer's request to the hold is

$$ t_{\text{hold}} \approx t_{\text{transcribe}} + t_{\text{AI check}} \approx 0.5\,\text{s} + 2.5\,\text{s} \approx 3\,\text{s}. $$

How well it works

Across 214 normal-call test runs, Scout never once rated a normal call "likely scam".

Challenges

• Speed. Our first transcriber only produced text when the caller paused, so warnings arrived after the call ended. Forcing phrases out every 4 seconds, moving to streaming transcription, and checking after every phrase brought the hold down to about 3 seconds after the ask.
• AI voices over the phone are hard to catch. Phone audio strips out much of what deepfake detectors look for. Single detectors missed live clones, so we combined three and made the voice check strengthen a scam decision rather than make it alone.
• False alarms. A protective tool that interrupts your doctor's office is a tool people turn off. We tested against real fraud alerts, debt collectors, pharmacies and family asking for money, and raised the bar to one clear rule: hold at 85%.
• Telephony. Call forwarding created ring loops, one carrier rate-limited us mid-test, and iOS labels every app-delivered call "Scout Audio." On our last day we also chased intermittent silent calls on the iPhone and added connection logging to find the cause.
• Privacy. Live call monitoring can feel like Big Brother. Scout saves no audio, deletes the transcript of any call where nothing was flagged, opts out of AI training, and shows Sound CU staff a member's calls only if the member opts in.

What we learned

• Judge the request, not the words. Scammers avoid trigger words; what gives them away is asking an unverified caller's target to move money, read back a code, or keep a secret.
• The best moment to stop fraud is before the transfer, while the member is still on the phone.
• Trust is a feature. A clear rule, a private notice, and the member always having the final choice matter as much as accuracy.

What's next

A 12-month plan with Sound CU: move the call engine to production cloud hosting, bring Scout into the Sound CU app on iPhone and Android, connect transfer freezes to the core banking system, and run a staff pilot followed by a 500-member opt-in pilot.

Built With

• claude
• github
• xcode