Inspiration
A year ago, a teammate's mom lost \$5,000 to an impersonation scam over the phone. Since then the threat has changed: it's no longer just a stranger with a script. With ten seconds of someone's voice, AI can clone it well enough to fool family.
The HuskyHack challenge data showed the same pattern. In family-emergency scams, money moves a median 12 minutes after the scam starts, but the first intervention comes at 16 minutes. Generic pop-up warnings were shown in 12 scam journeys and prevented none of them. Victims told their credit union "yes, I intended this" 44 times, because the scammer was coaching them.
The payment screen is too late. We wanted to stop the scam while it's still a phone call.
What it does
Scout listens quietly in the background of a member's normal calls from unknown numbers. The phone rings like any call, with the real number, even when locked, and the keypad still works. Scout runs two checks that keep each other honest:
- What is said: live speech-to-text in English and Spanish, read by Claude for 7 scam tactics, such as secrecy, "safe account" transfers, gift cards and requests for codes.
- How the voice was found: ***propriatary
If either check fires, Scout puts the caller on hold, not the member, and only the member hears why. They choose Back to call, End call or Call Sound CU. Sound Credit Union staff see high-risk calls live in an employee portal and can send a private message, join the call or end the scammer's line. The same analysis also checks scam texts and phishing emails.
How we built it
- Calls: SignalWire carries the call. Twilio, through a SIP bridge, rings our iPhone app with CallKit and PushKit, so it looks like a native call.
- Call engine: a Python/FastAPI service running Vosk for transcription and Claude Opus 5.5 for analysis.
- Website and portal: Next.js on Vercel, with Supabase for data, sign-in and the live updates.
- Decision rule. With $p$ as Claude's scam confidence and $v$ as the voice flag, Scout holds the caller when
$$ \text{hold} = \big(p \ge 0.90\big) \;\lor\; v_{\text{synthetic}} $$
A reminder plays at $t = 20\,\text{s}$, and if the member hasn't chosen by $t = 60\,\text{s}$, Scout drops the scammer.
- Insights page: we turned the challenge data (7,471 events, 118 members) into an employee dashboard with drill-down to each member's journey. Scam payments went to a brand-new payee 92.3% of the time, versus 4.0% for routine payments.
- Cost: about 2¢ of Claude per call and about half a cent per message check.
What we learned
- Timing beats accuracy. A perfect warning at minute 16 loses to a good one at minute 3. In our tests the median warning came at the caller's 3rd turn.
- Two signals are better than one. A company's phone menu has a computer voice but asks for nothing risky, so it isn't flagged. A cloned grandchild can sound perfect, but the voice screener still catches it.
- Privacy has to be designed in at every step: consent before anything is turned on, a consent notice for every caller (Washington requires it), audio never recorded, and staff seeing a call only when the member chooses.
- Members must always keep the last word. Every hold shows its reason and can be undone with one tap.
Test results:
| Test | Result |
|---|---|
| Scam conversations warned | 49 / 50 |
| False warnings on legitimate calls | 0 / 50 |
| Messages judged correctly | 30 / 30 |
| Cloned voice flagged on a live call | about 3 s after the caller started |
Challenges we faced
- Getting a phone number. Twilio wouldn't sell us a U.S. number without business verification. We built a hybrid: SignalWire owns the number, and a SIP bridge into Twilio rings the app.
- Callers hanging up. The member's phone rang about 15 seconds late, and the caller hung up. We now ring the member the moment the call arrives, and play a ringback tone meanwhile.
- Warnings that never arrived. On locked phones, the phone company never played our private announcements. The app now speaks warnings itself, and they travel inside the call as Twilio call messages.
- Cloned voices. An open-source detector we tried missed an ElevenLabs clone completely. A dedicated screener caught it within seconds on a live call.
- Phone menus. Early versions flagged "press 1 for billing". We added automated-menu detection and passed the member's keypad presses through to the menu.
- Legal consent. Washington is an all-party consent state, so every caller hears a short notice that the call is protected.
What's next
- A 90-day pilot with Sound Credit Union: three weeks of watching only (logging what Scout would do, with no holds), then live with opted-in members.
- Targets: fewer than 2% of holds on calls later found legitimate, and opt-in in under a minute.
- Expanding text and email protection.
Log in or sign up for Devpost to join the conversation.