ScamShield AI: Real-Time Fraud Detection

Inspiration

As digital payments, UPI transactions, and online communication continue to grow, cyber scams have become one of the biggest threats to internet users. Every day, people receive phishing messages, fake websites, scam voice calls, and fraudulent WhatsApp forwards that attempt to steal personal and financial information.

We wanted to build an intelligent assistant that not only detects scams but also explains why something is suspicious. By leveraging the Gemini API alongside machine learning and speech processing, we aimed to create a real-time cybersecurity companion that helps users make safer decisions before they become victims of fraud.

What it does

ScamShield AI is an AI-powered fraud detection platform that analyzes multiple types of suspicious content in real time.

Key Features 🔗 Detects phishing and malicious URLs 💬 Analyzes SMS, emails, and WhatsApp messages 🎙️ Converts suspicious voice recordings into text using Whisper 📷 Extracts text from screenshots using OCR 🤖 Uses the Gemini API to explain why content is suspicious in simple language 📊 Generates a risk score (0–100) with personalized cybersecurity recommendations 🛡️ Provides an intuitive dashboard for scam analysis and awareness

How we built it

Frontend React.js Tailwind CSS Framer Motion Backend FastAPI Python Uvicorn Artificial Intelligence Gemini API for contextual scam analysis, explainable AI, and personalized safety recommendations DistilBERT for text classification Scikit-learn for scam risk prediction Whisper for speech-to-text conversion EasyOCR for extracting text from screenshots Additional Components URL analysis using Python REST APIs connecting frontend and backend Git & GitHub for collaboration Workflow User uploads a message, URL, screenshot, or voice recording. The system extracts and preprocesses the data. AI models analyze scam indicators. Gemini generates a human-readable explanation and safety advice. ScamShield AI returns a risk score with recommended actions.

Challenges we ran into

Integrating multiple AI services into a single workflow. Designing an accurate scam scoring system for different input types. Processing voice recordings and screenshots efficiently. Handling prompt engineering to ensure Gemini produced reliable explanations. Building a clean and responsive user interface within the limited hackathon timeframe. Optimizing API response time for near real-time analysis.

Accomplishments that we're proud of

Built a unified AI platform capable of analyzing text, URLs, screenshots, and voice recordings. Successfully integrated the Gemini API to generate explainable cybersecurity insights instead of only binary predictions. Developed a clean, responsive dashboard with multiple scam detection modules. Combined traditional machine learning with generative AI to improve user understanding and trust. Created a scalable architecture that can be extended to browser extensions and mobile applications.

What we learned

Through this project, we gained practical experience in:

Building AI-powered web applications using FastAPI and React Integrating the Gemini API into real-world cybersecurity use cases Prompt engineering for explainable AI Speech recognition with Whisper OCR using EasyOCR Deploying full-stack AI applications Team collaboration and rapid prototyping during a hackathon

Most importantly, we learned how generative AI can transform complex cybersecurity analysis into clear, actionable advice that anyone can understand.

What's next for ScamShield AI: Real-Time Fraud Detection

Our vision is to transform ScamShield AI into a comprehensive digital security platform.

Future Roadmap 📱 Android and iOS mobile applications 🌐 Chrome and Edge browser extensions ☎️ Live scam call detection during phone conversations 🌍 Multi-language support for Indian regional languages 🧠 Fine-tuned scam detection models trained on Indian fraud datasets 🏦 Integration with banking and UPI security systems 👥 Community-powered scam reporting and threat intelligence 📈 Real-time cybersecurity analytics dashboard 🔍 Deepfake voice detection for AI-generated scam calls

Our ultimate goal is to empower every internet user with an intelligent AI assistant that prevents scams before financial loss occurs, making the digital world safer for everyone.

Built With

• api
• css
• distilbert
• easyocr
• fastapi
• framer
• gemini
• motion
• python
• react.js
• scikit-learn
• tailwind
• whisper