Inspiration

Scam texts and calls impersonating banks, couriers, and even police (the "digital arrest" scam) are an everyday reality in India right now, and most people only have a gut feeling to go on when one lands in their inbox. The tools that claim to help are usually just an LLM pattern-matching against its training data — confident-sounding, but not actually checking anything about the specific sender in front of you. We wanted to build something closer to what an actual fraud analyst does: pull out the concrete claims in a message, run deterministic checks against them, go look up real, current evidence about the specific domain/number/sender involved, and only then render a judgment — one that says why, not just high risk.

What it does

Vigil takes a suspicious message in whatever form it actually arrives in — pasted text, a screenshot of an SMS/WhatsApp forward, or a suspicious call (as audio or a typed-out transcript) — and runs it through a four-stage AI pipeline:

  1. Extraction — pulls out structured facts: claimed sender, URLs, phone numbers, payment/info requests, urgency language, and a first-pass scam-category guess.
  2. Heuristics — deterministic, non-AI checks against those facts: lookalike/typosquat domains, URL shorteners, suspicious phone number patterns, known-legit domain matches.
  3. Live web evidence — searches the web for real, current evidence tied to the specific sender/domain/number, kept deliberately separate from generic "this brand is commonly impersonated" background noise.
  4. Verdict — a reasoning model weighs all three stages together and produces a risk level (high/medium/low/uncertain), a confidence score, cited red flags, reassuring signals where they genuinely exist, a plain-English recommended action, and a category label. The result is a downloadable, shareable report card, and every analysis feeds a case log and trends dashboard.

How we built it

The backend is FastAPI/Python. Extraction uses Nemotron Nano on Nebius's Token Factory for fast, cheap structured output; the final verdict uses Nemotron 3 Super, a heavier reasoning model, so judgment is only ever rendered once all the evidence is in. Tavily supplies live web search for the evidence stage, split explicitly into entity-specific results versus generic category background — a deliberate architectural choice, not just a UI grouping, to stop generic "X is commonly impersonated" articles from inflating the risk score on their own. Heuristics are plain deterministic code (no API calls) for domain/phone/URL pattern checks, so they're instant and easy to debug. For input, Tesseract OCR handles screenshots and faster-whisper handles call-audio transcription, feeding into the exact same pipeline as pasted text. The frontend is hand-built HTML/CSS/JS with no framework, which gave us full control over the visual identity. Everything ships as one Docker image deployed as a single Google Cloud Run service, so the static frontend and the API are served from one URL with nothing for a judge to configure.

Challenges we ran into

False positives from background noise. Early on, a verified-legit, zero-red-flag message could still get pushed to medium/high risk because Tavily returned generic articles about that brand being a common impersonation target. We fixed this by splitting evidence into "specific to this sender" versus "generic category background" and explicitly instructing the verdict model that background alone can never move the risk level on its own. LLM structured-output reliability. The reasoning stage would occasionally return JSON missing required fields, or get stuck in a repetition loop and hit its token limit, especially on messy input like a poorly-OCR'd screenshot. We added automatic retries with explicit correction prompts, a mild repetition penalty on the extraction model, and safe fallback defaults so one bad model response degrades gracefully instead of crashing the whole analysis. Ephemeral storage on Cloud Run. The case log and trends dashboard reset on a cold start or redeploy since there's no persistent database yet — currently mitigated with labeled seed data as a fallback. Getting OCR and audio transcription into one container. Tesseract and faster-whisper both needed real system dependencies (tesseract-ocr, ffmpeg) alongside the Python stack, which took some Dockerfile iteration to get building reliably on Cloud Build.

Accomplishments that we're proud of

• A genuinely multi-modal detector — text, screenshot, and call audio all feed the same evidence-grounded pipeline, not three separate bolted-on features. • A verdict stage that's actually grounded in live, specific evidence rather than an LLM guessing from memory — including catching and fixing a real false-positive regression where generic background noise was overriding a verified-legit sender. • A demo-ready deploy behind a single URL, with the static frontend and API served from one Cloud Run service and zero manual setup for anyone opening the link. • A visual identity that doesn't look like a templated hackathon project.

What we learned

Structured LLM output needs to be treated as unreliable by design, not as an edge case — retries, validation, and safe defaults aren't optional polish, they're what keeps a demo from crashing in front of a judge. We also learned how easily a scam-detection model can be fooled into false positives by evidence that's real but not actually about the message in front of it, which pushed us toward a much more deliberate split between specific and generic evidence than we originally planned. On the infra side, we got a much better feel for what Cloud Run's stateless-container model costs you (ephemeral storage, cold starts) versus what it buys you (one command to deploy, zero server management).

What's next for Vigil

• Persistent storage (Firestore or similar) so the case log and trends dashboard survive restarts and actually accumulate real data over time. • A one-tap mobile share-sheet or browser extension so people can send a screenshot straight to Vigil without leaving their messaging app. • Local-language support (Hindi and other regional languages), since many of the scam texts and calls this targets don't arrive in English. • A lightweight feedback loop — letting users confirm a verdict was right or wrong — to sharpen the heuristics and category labels over time. • Bulk/community mode for moderators and support teams triaging many reports at once.

Built With

Share this project:

Updates

Submission history