Sentinel

Forward a suspicious invoice or voice note into Slack, tag @Sentinel, and watch specialist agents cross-examine each other's findings to catch fraud contradictions no single check would spot alone.

Comment

Inspiration

Fraud doesn't get caught because one signal is wrong — it gets caught because two signals disagree. A forged invoice can pass an OCR check and a cloned voicemail can pass a voice check, but they rarely agree with each other under cross-examination. We wanted to build something that lives where finance and ops teams already work — Slack — and does exactly that: cross-examine evidence instead of scoring it in isolation.

What it does

You forward an invoice, CSV, or voice note into a Slack channel and type @Sentinel investigate. Sentinel immediately ACKs (under 5ms) and kicks off eight specialist agents in the background — finance, NLP, vision/OCR, threat-intel, stylometric, voice authenticity, compliance, and policy — each producing independent claims about the evidence. A deterministic 3-way contradiction engine then reconciles those claims across three axes (visual vs. structured data, tone, and voice authenticity) and looks for disagreements a single check would miss. The result is a verdict — CLEAR, REVIEW, FRAUD_LIKELY, or NEED_MORE_INFO when evidence is too thin to call — posted back to the thread as a Block Kit card, with a live React console showing the investigation unfold via SSE, a blast-radius graph of who else is exposed, and a self-writing detection rule drafted from the case for next time.

How we built it

  • Gateway: FastAPI + Slack Bolt, with Slack event ACKs handled in under 5ms and the actual investigation run out-of-band as a background task, so we never hit Slack's 3-second timeout.
  • Contradiction engine: a deterministic, symbolic reconciler — no LLM guesswork — that compares claims from different agents pairwise and flags disagreements across axes.
  • Agents: eight specialists that harvest claims from evidence. Most are deterministic (regex, pdfminer text extraction, finance math, acoustic feature detection) rather than LLM calls, by design — for reliability, testability, and zero hallucination risk on stage. Threat-intel is the one agent hitting a real external API (VirusTotal).
  • Persistence & trust: PostgreSQL for case lifecycle and evidence, plus a hash-chained audit log so every verdict is independently verifiable after the fact.
  • Graph layer: a NetworkX-backed cache fed from the Slack Event API, used for BFS blast-radius mapping, PageRank-based campaign clustering, and active quarantine of compromised threads.
  • Learning loop: cases automatically synthesize shadow detection rules, which get promoted to enforced rules once they prove out — so a pattern caught once gets caught instantly next time.
  • Interfaces: Slack Block Kit for the in-Slack experience, a React investigation console for a live, visual view of the pipeline, and a full MCP server exposing Sentinel's tools/data to other agents.
  • Evaluation: a labeled 42-case dataset and harness measuring precision/recall/F1 with ablation, so we can show — not just claim — that the contradiction engine adds signal beyond any single agent.

Challenges we ran into

Getting Slack's 3-second ACK window to coexist with a genuinely multi-agent investigation pipeline meant committing early to an async, out-of-band worker architecture rather than bolting it on later. We also hit a real gap between our in-memory dev backend and the Postgres backend drifting out of sync as the schema evolved — a good reminder that dual-backend systems need parity tests, not just parity intentions. And we made a deliberate, disclosed tradeoff: most specialist agents are deterministic rather than LLM-backed, which we chose for demo reliability and testability, but which required being upfront about rather than letting the "multi-agent" framing overstate what's happening under the hood.

Accomplishments that we're proud of

The 3-way contradiction engine and the evaluation harness are the two things we're proudest of — they're deterministic, fully tested, and prove the core thesis with real precision/recall numbers rather than a demo that just "looks" convincing. We're also proud of shipping a hash-chained audit log, a self-writing rules engine, and a full MCP server alongside the core investigation flow — this isn't just a fraud checker, it's a system built to be trusted and to keep improving.

What we learned

That the hardest part of a "multi-agent" system isn't spinning up agents — it's designing the reconciliation layer that makes their disagreements meaningful. And that being explicit about what's simulated (federated pattern network, honeypot agent) versus what's deterministic-by-design (most specialist agents) versus what's genuinely learned (the rule synthesizer) makes for a stronger, more defensible pitch than pretending everything runs on an LLM under the hood.

What's next for Sentinel

Wiring real LLM calls into the specialist agents where it adds genuine value (not just narrative), implementing Slack Canvas case files for a persistent per-case record, expanding the voice authenticity model beyond a single curated validation pair, and moving temporal recall from a lightweight hash-based embedding to a proper pgvector + sentence-transformers setup.

Built With

  • ai
  • cybersecurity
  • digital-forensics
  • enterprise-security
  • fastapi
  • generative-ai
  • graph-analytics
  • incident-response
  • llm
  • mcp
  • multi-agent-ai
  • networkx
  • ocr
  • postgresql
  • psycopg
  • python
  • rag
  • risk-management
  • slack-api
  • tailwindcss
  • threat-intelligence
  • vector-search
  • vite
  • voice-deepfake-detection
Share this project:

Updates