Devil's advocate

Issues
Issues
Timeline
Breakdown
Avatar

Inspiration

The greatest risk in a team is miscommunication, but when the team grows not everyone has the time to read every threads, every messages.

So I wanted to build a agent that can make sure everyone in the team can be up to date regarding a subject in a matter of seconds.

So that Slack teams never have to guess important decisions again.

What it does

Devil's Advocate is a Slack agent that red-teams your decisions using your team's own past messages.

You give it a decision: "We ship Apple Pay to production tomorrow. Any objections?" and it hunts down the counter-arguments buried in your own channels: the bug flagged three weeks ago, the customer complaint, the mandated staged rollout everyone forgot.

Plans a hunt. Claude turns your decision into semantic questions and literal keyword searches across the angles that kill launches: technical, customer, timing, past incidents. Searches your real history with the Slack Real-Time Search API, permission-aware, so it only sees what you can see.

Chases its own leads. A second, agentic pass reads what it found, deduces one un-explored contradictory thread (an incident, a freeze, an error code) and hunts it down.

Cross-checks a second system corroborating GitHub issues and PRs via an MCP server. Delivers a verdict: a risk score with a color-coded tier, objections ranked by severity, evidence cards that link to the real Slack thread, a "regret timeline" replaying what your team already knew, and a "Convince me anyway" button that flips the advocate to argue for. It answers in your language.

The citations can't be hallucinated because Claude only chooses which retrieved message backs an objection, by array index. The quote, author, channel and permalink are rendered straight from the search response, never from the model. The app literally cannot invent a source, and every card links to proof you can click.

How we built it

@slack/bolt v4 (Socket Mode): Assistant surface, app_mention, App Home, interactive buttons.
Slack Real-Time Search API (assistant.search.context): semantic + keyword fan-out, dedupe, permission-aware via action_token. This is the load-bearing capability.
MCP: a Model Context Protocol server exposing GitHub issues/PRs as a second, independent evidence source.
Anthropic Claude via forced tool-use, so every query plan and verdict comes back as structured JSON.
TypeScript + tsx, in-memory context store for button state.

Challenges we ran into

Keeping the model honest. Early versions let Claude write the citations, which meant it could subtly paraphrase or invent them. We reworked the pipeline so Claude only returns an index into the real search results, and the renderer pulls the quote, author and link straight from the response. A wrong index can only ever cite the wrong real message, never a fake one.
Semantic versus keyword search. Natural questions trigger semantic search, but product names and error codes only surface with literal keyword search. We fan out both and dedupe the results, which is what actually lets the two pass hunt find the buried threads.
A judge should not need to read French. We built a language layer: the agent detects the language of the decision and renders the whole experience, verdict, buttons, progress and errors, in that language. You can demo in French and have a judge test it in English, on the same app.
No dead ends. An empty workspace, an empty mention, or a decision with zero contradictory evidence each needed a deliberate, honest state instead of a broken one. When nothing is found, the agent shows a clearly labeled generic premortem checklist rather than faking severity.

Accomplishments that we're proud of

Shipping the agent is already everything, I joined this hackaton a bit skeptical because I thought I wouldn't have the courage to keep pushing until everything is submited.