-
-
Elenchus Agentic RAG Pipeline: Ingests via Socket Mode, filters with Groq, fetches MCP data, and executes LLM-as-a-judge verification.
-
Automated Contradiction Detection: Elenchus silently monitors channels and drops an evidence-backed card when a false claim is detected.
-
Anti-Spam by Design: When a claim is true, the LLM judge verifies it in the background and intentionally stays silent to reduce noise.
-
Manual On-Demand Verification: Users can force a real-time fact-check on any Slack message using our native custom UI shortcut.
Inspiration
Most workplace AI in Slack today is reactive. Someone asks a question, the bot searches for an answer, and hopes it found the right thing. That model has a real weakness: it can only correct a wrong belief if someone thinks to ask. We've all seen it happen, someone posts "the bug is fixed" in Slack, everyone moves on, and two hours later it turns out it wasn't actually fixed.
Slack's own help documentation even acknowledges that Slackbot and Slack AI can hallucinate, generate answers that sound confident but are factually wrong, and their current guidance for catching this is manual: ask again, keep conversations short, or provide more specific sources yourself. There's no automated check for this. Elenchus is that missing layer.
What it does
Elenchus monitors a Slack channel in the background and watches for status or decision claims people make in passing, "we deployed payments yesterday," "the migration is done." Instead of waiting to be asked, it checks the claim itself:
- Classifies whether a message is actually a status/decision claim worth checking, so it stays quiet on small talk and general chat.
- Gathers evidence from recent Slack conversation, a connected GitHub/deployment data source over a real MCP (Model Context Protocol) connection, and, when the bot is directly @-mentioned, Slack's Real-Time Search API for workspace-wide context.
- Judges whether the evidence supports, contradicts, or can't confirm the claim, using an LLM-as-judge approach, with a confidence estimate.
- Speaks up only when something's wrong. If the claim holds up, Elenchus says nothing. If it doesn't, it posts a card directly in the thread showing the claim, the evidence, and a confidence level, right where the wrong claim was made.
There's also a manual path: right-click any message → "Verify with Elenchus" → get the same evidence-based check on demand, for any message including a Slack AI answer.
How I built it
The verification logic is built on earlier work on hallucination detection in LLMs, specifically LLM-as-judge verification and uncertainty estimation, repositioned as the entire product rather than a side feature. The stack is Slack Bolt on Socket Mode, Groq (Llama 3.1) for classification and judging, a real MCP server/client for the evidence connector, and the Real-Time Search API for @-mention-triggered workspace search. It's deployed on Render with a scheduled job keeping it alive continuously.
Two design choices came from looking at actual documented Slack complaints rather than assumptions: evidence citations link to real permalinks instead of quoted text (addressing the common complaint that finding an old message requires already knowing what to search for), and a 30-minute cooldown prevents re-flagging the same topic repeatedly (addressing notification fatigue, the most-cited Slack complaint by volume).
Challenges faced
Getting a genuine MCP protocol connection working (not just a same-process function call standing in for one), and discovering that Slack's Real-Time Search API only issues the required action token when the bot is directly @-mentioned, not for passive channel monitoring, which shaped how RTS gets used versus the always-available Slack history path.
What's next
Swapping the seeded GitHub/deployment data behind the MCP connector for a live production source - the protocol connection is already real, so this is a data-source change, not an architecture change.
Log in or sign up for Devpost to join the conversation.