GuardianEye

Real-time scam detection for suspicious screenshots, calls, and messages.

Comment

Inspiration

GuardianEye was born from a simple but urgent problem: scams are getting smarter, and the people most vulnerable, especially seniors, often can't tell the difference between a real message and a sophisticated phishing attempt. With AI-generated deepfakes, spoofed caller IDs, and convincing phishing pages becoming the norm, we wanted to build something that could fight AI-powered scams with AI. The goal: let anyone upload a suspicious screenshot, voice call, or text message and get an instant, trustworthy verdict.

What It Does

GuardianEye is a multi-modal AI scam detection platform. Users submit suspicious content, a screenshot of a phishing email, a recorded phone call, or a sketchy text message, and within seconds receive a RED / YELLOW / GREEN verdict with a confidence score, red flags, and actionable recommendations. Behind the scenes, a multi-agent pipeline runs in parallel: computer vision extracts text and logos, NLP identifies entities (phone numbers, URLs, company names), search agents check scam databases in real time, and a graph database tracks how scam entities connect across reports, building a living network of fraud intelligence.

How We Built It

  • Frontend: React 19 + TypeScript with Vite, styled with Tailwind CSS and Shadcn/ui. Real-time progress is streamed to the browser via Server-Sent Events (SSE) so users see each analysis step as it happens.
  • Backend: FastAPI (Python) with a fully async architecture. The scan pipeline orchestrates multiple AI services in parallel, Reka Flash and Gemini for vision, Modulate Velma-2 for voice emotion analysis, GLiNER for entity extraction, Tavily for fast reputation checks, and Yutori for deep autonomous research across 8+ scam databases.
  • Graph Intelligence: Neo4j Aura stores scam reports as a knowledge graph, linking phone numbers, URLs, and impersonated companies across submissions. This reveals scam networks, when one phone number appears in 50 reports, that's signal.
  • Autonomous Scouts: Yutori-powered agents run continuously in the background, monitoring the FTC, FBI IC3, Reddit r/Scams, ScamAdviser, and AARP Fraud Watch for emerging threats every 30 minutes.
  • Caching: SHA-256 content hashing with SQLite + Turso cloud sync ensures identical submissions return cached results instantly and persist across deploys.

What We Learned

  • Graceful degradation is everything. Every external API can fail. We built 4-5 layers of fallbacks (Reka to Gemini, Modulate to Gemini, GLiNER to regex, LLM verdict to rule-based scoring) so the system always returns a result, even when services are down.
  • Two-tier analysis, delivering a fast verdict in 3-5 seconds while continuing deep research in the background, was critical for user experience. Nobody wants to wait 90 seconds staring at a spinner.
  • Graph databases change how you think about fraud. Storing entities as nodes and relationships revealed patterns that flat databases never could. A single phone number connecting 12 "IRS" scam reports tells a story.
  • SSE streaming makes AI pipelines feel alive. Showing users each step in real time (vision complete, entities extracted, database checked, verdict ready) builds trust and transparency.

Challenges

  • API reliability and latency. Orchestrating 5+ external AI services with unpredictable response times required careful timeout management (e.g., 15-second Reka timeout with automatic Gemini fallback) and parallel execution to keep total scan time under control.
  • Cache coherence with SSE replay. Cached scans needed to replay the same SSE event stream with staggered delays to maintain the real-time animation feel, even though the result was already known.
  • Neo4j data modeling. Designing a schema that could connect entities across scans while supporting efficient network traversal queries took several iterations.
  • Multi-modal input handling. Supporting screenshots (with client-side image compression), audio files, and raw text through a unified pipeline, each with different AI services and processing steps, added significant architectural complexity.

Built With

Share this project:

Updates