Guardian

Guardian detects scam calls in real time, tracks the psychological pressure being used against your loved one, and alerts your family before the money is gone.

Comment

Inspiration

Every year, Americans lose over $10 billion to phone scams, and elderly adults are disproportionately targeted. We kept hearing stories — a grandparent tricked into buying gift cards by someone pretending to be the IRS, a retiree wiring their savings to a fake "bank fraud department." The worst part? By the time a family member finds out, the money is already gone.

We asked ourselves: what if we could detect a scam while the call is still happening? Not after the fact, not through a post-call review — but in real time, before any damage is done. That's how Guardian was born.

What It Does

Guardian is an iOS app that acts as a real-time scam shield during phone calls. Here's how it works:

  1. Call detection — When an incoming call arrives, Guardian detects it via CallKit and prompts the user to start scanning.
  2. Live transcription — The user puts the call on speaker. Guardian listens through the device microphone and streams audio to Deepgram for real-time speech-to-text.
  3. Instant keyword flagging — As words are transcribed, Guardian checks for known scam phrases ("IRS," "gift cards," "arrest warrant," "pay immediately") and flags them instantly with red badges.
  4. AI-powered analysis — Every few seconds, the accumulated transcript is sent to Claude (Anthropic) via a Supabase Edge Function. Claude analyzes the conversation for scam patterns: pressure tactics, impersonation, suspicious payment requests, personal info harvesting.
  5. Immediate alert — If a scam is detected, Guardian fires a full-screen warning with haptic feedback, telling the user to hang up immediately.
  6. Family notification — A Twilio SMS is automatically sent to a designated family member with a summary of the detected scam, so someone else can intervene.
  7. Family dashboard — Family members can monitor scam events in real time through a live dashboard powered by Supabase Realtime.

How We Built It

Mobile App — React Native CLI (no Expo) targeting iOS. We needed native module access for CallKit call detection, audio session management, and in-call audio routing via react-native-incall-manager.

Speech-to-Text — Deepgram's streaming WebSocket API with diarization enabled. Audio is captured at 16kHz via react-native-live-audio-stream, encoded as linear16, and streamed directly to Deepgram. Diarization lets us separate speakers so we can focus on the caller's words, not the user's.

Scam Detection (Keyword Layer) — A local keyword engine runs on every transcript update with zero latency. It checks for 30+ known scam phrases across categories: urgency, impersonation, payment methods, remote access, and personal info requests. A visual threat meter escalates from "No Threats" → "Low Risk" → "Suspicious" → "Scam Detected" as more keywords are flagged.

Scam Detection (AI Layer) — Claude analyzes transcript chunks through a Supabase Edge Function. The prompt is tuned to detect six scam categories and returns a structured JSON response with a confidence score, flags, and a human-readable summary.

Backend — Supabase handles everything server-side: Postgres for call/transcript storage, Edge Functions for Claude and Twilio integration, Realtime for pushing scam events to the family dashboard, and auth for user management.

Alerts — Twilio sends SMS alerts to the user's designated family contact when a scam is detected. The alert includes a summary of what was detected.

Architecture

Device Microphone
       │
       ▼
React Native App ──stream audio──▶ Deepgram (STT)
       │                                │
       │                         transcript text
       │                                │
       ▼                                ▼
  Keyword Engine ◀──────────────────────┘
  (instant flags)        │
       │                 ▼
       │        Supabase Edge Function
       │              │
       │              ├──▶ Claude API (scam analysis)
       │              ├──▶ Supabase DB (log event)
       │              ├──▶ Twilio SMS (family alert)
       │              └──▶ Supabase Realtime (dashboard)
       ▼
  Threat Meter + Alert UI

Challenges We Faced

iOS audio during active calls. This was by far the hardest technical challenge. iOS aggressively locks the microphone during cellular calls. We tried multiple approaches: manual AVAudioSession configuration, custom native modules with overrideOutputAudioPort(.speaker), timed delays for audio route propagation, and even a Twilio conference call bridge approach. We ultimately used react-native-incall-manager, a package built specifically for managing audio during calls, which handles the AVAudioSession lifecycle and forces speaker output correctly.

WebSocket timing. Our initial Deepgram integration dropped the first few seconds of audio because sendAudio() was called before the WebSocket connection was fully established. We had to refactor the connection to return a proper Promise, add a pending audio buffer, and flush it on onopen.

Speaker diarization filtering. Deepgram transcribes both the user and the caller. We implemented confidence-based speaker identification — the user's voice (closer to the mic) has higher transcription confidence, so we filter it out and show only the remote caller's words.

Twilio trial restrictions. During development, we discovered Twilio trial accounts only accept calls from verified phone numbers, which blocked our conference bridge testing. Debugging this required inspecting Twilio call logs via their API to find error code 21264.

What We Learned

  • iOS audio session management is one of the most underdocumented and frustrating parts of iOS development — the behavior changes between cellular, VoIP, and speaker modes in ways Apple doesn't clearly explain.
  • Real-time AI analysis during a phone call is surprisingly feasible with the right architecture — streaming STT + periodic LLM analysis gives sub-10-second detection latency.
  • A two-layer detection approach (instant keywords + deeper AI analysis) gives the best of both worlds: zero-latency alerts for obvious scam phrases, and nuanced detection for sophisticated social engineering.
  • Building for elderly users forces better design for everyone — large tap targets, clear language, and unmissable alerts are just good UX.

What's Next

  • Twilio conference bridge for true in-call audio capture (bypasses iOS mic restrictions entirely by routing call audio through a server)
  • Live Activity / Dynamic Island integration for one-tap scanning from the lock screen
  • Scam pattern learning — aggregate anonymized data across users to identify emerging scam scripts
  • Multi-language support — scams target non-English speakers too
  • Android version — expand protection beyond iOS

Built With

  • React Native
  • Swift
  • Deepgram
  • Claude (Anthropic)
  • Supabase
  • Twilio
  • Node.js
  • CallKit

Built With

Share this project:

Updates

Submission history