Scam Shield

AI that stops scams before they happen

Comment

Inspiration

Scam calls and online fraud are becoming increasingly common, and many people lose money because they realize the scam only after it is too late. Scammers exploit trust, urgency, and social engineering tactics to manipulate victims in real time.

One of the biggest concerns is that elderly people are the most vulnerable. Many of us have parents or grandparents who may not be familiar with modern digital scams. A convincing phone call pretending to be from a bank, courier service, or government authority can easily trick them into sharing sensitive information or sending money.

Scam detection is also difficult because scammers constantly change their identities and tactics. They may impersonate banks, tech support, government officials, or online buyers. In addition, many communication platforms use end-to-end encryption, which makes it harder for traditional systems to analyze conversations and detect fraud.

Our goal was to build a system where AI acts as a real-time guardian, monitoring conversations, identifying scam behavior, and protecting the user before they share sensitive information or send money.

What It Does

Our project is an AI-powered scam protection system that monitors conversations and app activity in real time.

The system can:

  • Listen to conversations and detect scam patterns
  • Calculate a dynamic risk score as the conversation progresses
  • Show warnings when suspicious behavior appears
  • Block dangerous interactions when the risk becomes high
  • Analyze URLs and detect phishing or malicious links
  • Provide a dashboard showing scam reports and AI explanations

The Android app acts as the user’s protection layer, displaying:

  • Low-risk warnings through notifications
  • High-risk alerts through a blocking overlay
  • A dashboard with scam detection reports

For testing, we simulate real calls using LiveKit, while AI models analyze conversations using Google Gemini.

How We Built It

Our system consists of three major components:

1. Real-Time Communication Layer

We used LiveKit to create a real-time audio environment where participants can talk during simulated calls.

This allowed us to simulate realistic call scenarios while streaming the audio to the AI system for analysis.

2. AI Analysis Engine

We used Gemini Live API to process live audio streams and analyze the conversation context.

The AI looks for patterns such as:

  • authority impersonation
  • urgency pressure
  • requests for sensitive information
  • financial manipulation tactics

Based on these signals, the system calculates a risk score that increases as suspicious behavior appears.

3. Android Protection Layer

The Android application monitors activity and provides real-time protection.

The app:

  • displays warnings when suspicious behavior is detected
  • blocks high-risk interactions using an overlay
  • scans URLs for phishing or malicious content
  • shows a dashboard with scam reports and AI reasoning

Challenges We Ran Into

We encountered a challenge where the model was only able to listen to a single audio track at a time. However, in a real conversation the AI agent needs to hear both participants simultaneously to properly understand the interaction.

To solve this, we implemented an audio mixing layer that combines the audio streams from multiple participants into a single synchronized stream. This mixer merges the voices without overlapping or distortion, allowing the AI model to analyze the full conversation context in real time.

Another big challenges was detecting scams without triggering false alarms. Many normal conversations can contain words that appear suspicious, so we had to focus on detecting patterns of manipulation rather than individual keywords.

We also faced a limitation on Android where recording phone calls is heavily restricted for privacy and security reasons. Because of these platform restrictions, our app could not directly capture audio from regular phone calls.

To overcome this, we designed our own VoIP-based calling system inside the app. Instead of relying on the native phone call infrastructure, users communicate through our in-app calling interface. This allows us to securely stream the audio to the AI system in real time, enabling the model to analyze the full conversation and detect scam patterns while the call is happening.

What We Learned

Through this project we learned:

  • how real-time AI models can analyze live conversations
  • how scammers use psychological manipulation tactics
  • how to design a risk scoring system for behavioral detection
  • how to integrate AI with real-time communication systems

Most importantly, we learned that AI can move beyond passive analysis and actively protect users in real time, especially those who are most vulnerable such as elderly users.

Built With

Share this project:

Updates

Submission history