Guardian

Inspiration Phone scams cost Americans over $25 billion a year, and the victims are disproportionately elderly and vulnerable. We've all seen it — a grandparent tricked into sending gift cards, or a family member nearly fooled by a fake IRS agent. Existing solutions like carrier-level spam labels are passive. They warn you. We wanted to build something that acts — intercepting calls silently, making the decision, and protecting people who may not know to be suspicious in the first place.

How We Built It Guardian is a multi-layer pipeline engineered to handle the full spectrum of call threats:

• Layer 1 — Identity Verification: Incoming calls hit VoIP.ms, which validates the caller against STIR/SHAKEN attestation standards to catch spoofed numbers immediately.
• Layer 2 — Database Screening: The call is checked against a hand-built whitelist of 178,000 trusted numbers (hospitals, pharmacies, fire departments, police) and a blacklist of 64,000 reported scam numbers sourced from open fraud databases. Both databases were built with strict validation logic to prevent legitimate callers from ever being blocked. This layer alone resolves 90% of unknown calls in milliseconds.
• Layer 3 — AI Proxy Analysis: Numbers that pass screening enter a 10-20 second proxy call hosted through VoIP.ms at roughly $0.002 per call. The audio feeds into a fusion model — one DistilBERT-based classifier analyzing transcript context for scam language, and a second model detecting AI-generated voices. Both must clear for the call to connect.

The backend is built on Bun, Hono, and TypeScript. The ML layer runs PyTorch with FastAPI. The mobile app is built with Tauri v2, React, Rust, and Swift. Infrastructure runs on AWS.

Challenges Building the databases was the hardest part. Aggregating 178,000 verified safe numbers and 64,000 scam numbers from open-source data required building our own ingestion pipeline with validation rules strict enough to ensure no legitimate caller gets blocked. One wrong rule could mean someone's grandma never gets a call from her doctor. Cost constraints shaped the architecture. The AI proxy layer had to be cheap enough to run at scale. We designed the pipeline so that the vast majority of calls never reach the AI layer at all — the database checks act as a cheap, fast gate. By the time a call reaches the proxy, we're spending fractions of a cent on something that genuinely needs deeper analysis. Fusion model calibration. Getting two models — text and voice AI detection — to agree without creating too many false negatives required careful threshold tuning. Erring too far toward blocking means real calls get dropped. Too lenient and scams slip through.

What We Learned Building Guardian taught us that real-world security is never one thing — it's layered. No single check is reliable enough on its own. We also learned how underbuilt the public infrastructure around phone fraud is. The data exists, but it's scattered, inconsistent, and unmaintained. Building the databases ourselves was the most unglamorous part of the project and probably the most impactful. Clean data beats a fancier model every time.

Built With

• amazon-web-services
• bun
• distilbert
• fastapi
• hono
• javascript
• python
• pytorch
• react
• rust
• stir/shaken
• swift
• tauri
• typescript
• voip.ms