ScamShield

Webapp - Landing Page
Webapp - Stats
Webapp - Flow
Webapp - Features
Webapp - dashboard
Webapp - dashboard
Mobile App - Sign In
Mobile App - Alert screen
Academy

Project Story

It started with a phone call.

A recorded FedEx voice told me my package was held at customs - suspected drugs, legal action imminent, press 1 to speak to an officer. I knew instantly it was a scam. I hung up. But I couldn't stop thinking, what if I hadn't? What if it was my grandmother? The script was so polished, so confident, so designed to make a scared person comply before they could think.

That's what we built ScamShield to prevent.

We are three MSCS students who wanted to build something that mattered beyond a hackathon. Scam calls steal $25 billion from Americans every year and the victims are almost always elderly, non-technical, or simply caught off guard. Every tool today blocks calls before they connect. Nobody had built protection for what happens after the call goes through.

So we did.

ScamShield forwards your calls through our system. When an unknown caller connects, we stream both sides of the audio in real time to our detection engine: a two-layer system that combines a rule-based scorer for known scam patterns with Claude AI cross-referencing the live conversation against thousands of documented FTC scam scripts. Known contacts are whitelisted automatically so we never record a call from someone you trust. The moment a scam is detected, Twilio injects a voice warning directly into the call telling the user to hang up and sends you a push notification directly on the app.

The hardest technical challenge was the real-time audio pipeline. Streaming both legs of a live phone call, converting speech to text with low enough latency to catch a scam before damage is done, while keeping the detection accurate enough to avoid false positives on legitimate calls. That required careful architecture across Twilio Media Streams, Deepgram's streaming WebSocket, and a dual-layer detection engine.

We also built two additional features we're proud of. The first is a web dashboard that shows call trends, flagged keywords, timestamped transcripts as time series charts, real-time scam detection rates, and Claude's confidence scores, giving families and care organizations a full picture of the threat landscape targeting their loved ones.

The second is ScamShield Academy, an ElevenLabs-powered multilingual education layer where AI tutors teach users how to recognize scams, quiz them on red flags, and simulate real scam calls so people can practice hanging up before it counts. Every session generates a detailed report of what red flags were missed. Scores are saved so users can track their progress over time. Because the best protection is a person who already knows what a scam sounds like.

We learned that the hardest problems in this space aren't technical, they're about trust. Building a system that listens to phone calls requires earning the right to do so. Every architectural decision we made from whitelisting known contacts, never storing raw audio, making the alert a phone call and a push notification rather than a running background process, was made with that trust in mind.

Scams follow scripts. So does ScamShield.

What's next?

The one thing we haven't solved yet is iOS background protection right now the Twilio callback handles it, but a native always-on solution is the right long term answer. That's the next engineering sprint. Additionally, ScamShield right now catches scams after they call you. The next version catches them before. Every call we analyze adds to a growing dataset of scam scripts, phone numbers, and tactics. Patterns emerge and the same script hits thousands of people in waves, the same spoofed numbers rotate through regions, the same urgency phrases appear weeks before mainstream awareness catches up. We want to turn that data into a early warning system. When a new scam script starts appearing in our call stream, we flag it before the FTC does.