Hindsight - org memory for Slack

• 

  decision-record-hindsight

• 

  save-to-confluence

• 

  relay-incident-triage

• 

  pulse-voice-of-customer

• 

  onramp-onboarding

• 

  clarity-accessibility

• 

  quorum-meeting-closer

• 

  ledger-commitments

• 

  standup

• 

  dealroom-brief

• 

  lifeline-mutual-aid

Inspiration

Every team makes its best decisions in Slack — then loses them. A new hire asks "why did we choose Postgres over ClickHouse for analytics?" and nobody remembers; the engineer who ran the benchmark shipped it two months ago and moved teams. The answer is sitting in Slack, but search returns noise, not reasoning.

We kept watching the same quiet tax get paid over and over: onboarding dragging on for weeks, the same incidents re-debugged from scratch, and hard-won context evaporating the moment someone leaves. Slack is where the thinking happens — so we set out to turn it from a chat log into queryable organizational memory that answers with receipts.

What it does

Hindsight is a single Slack agent backed by ten specialized pipelines. You @Hindsight a question in natural language; an orchestrator routes it to the right pipeline, which retrieves the relevant Slack history, reasons over it with an LLM under hard citation enforcement, and replies with a clean Block Kit card where every claim carries a source permalink. Decision Records can be saved to Notion or Confluence in one click.

How we built it

All three of the challenge's Slack technologies are load-bearing — pull any one out and the product stops working:

• Real-Time Search API (assistant.search.context) is every pipeline's retrieval step — there is nothing to reason over without it. A progressive keyword-fallback layer keeps retrieval robust whether or not semantic search is enabled on the workspace.
• Slack AI / Agent surface is the interface and the router: the assistant container with suggested prompts, streaming status, and orchestrator routing (keyword-first, then LLM).
• MCP (Model Context Protocol) is the orchestration substrate. Each of the ten pipelines is registered as an MCP tool the orchestrator invokes over real JSON-RPC; persistence is Hindsight acting as an MCP client to external Notion / Confluence MCP servers. No orchestration, no durable memory without it.

The stack is TypeScript + Node.js on Slack Bolt (Socket Mode). Pipelines share three services — RTS retrieval, a pluggable LLM layer (Anthropic / OpenAI / Accenture AI Refinery behind one interface), and MCP persistence. Citation enforcement lives in grounded synthesis: it drops any rationale bullet that isn't backed by a retrieved source and forces confidence: low when nothing is grounded, so the agent physically cannot make an uncited claim. Following Slack's zero-data-copy policy, retrieved content is never stored — only user-approved records are persisted.

Challenges we ran into

• Making citations unfakeable. It's easy to ask an LLM to "cite sources"; it's hard to guarantee it. We moved provenance out of the prompt and into code — synthesis validates every [n] marker against retrieved messages and discards anything unsupported. No citation, no claim.
• Retrieval that works on any workspace. RTS semantic search needs a Slack AI–enabled workspace. We built a progressive fallback (full question → keywords) so demos and real installs stay robust either way.
• Keeping MCP real, not decorative. Instead of faking tools, we stood up an internal MCP server that turns the pipeline registry into tools automatically, plus an outbound MCP client for wiki persistence — real JSON-RPC on both sides.
• One agent, ten jobs. A single orchestrator had to route very different intents reliably. Keyword-first routing gives deterministic, demo-safe behavior; the LLM router handles the long tail.
• Provider portability. We put Anthropic, OpenAI, and Accenture AI Refinery behind one interface; AI Refinery rides the OpenAI SDK via a custom baseURL (no Python sidecar), so a single key can reach multiple model families.

What we learned

• Provenance is the product. The moment a rationale bullet becomes a clickable permalink to a two-month-old message, trust changes completely — that's what makes org memory usable.
• MCP is a clean orchestration substrate, not just a tool-calling gimmick: pipelines-as-tools kept the system modular and let persistence live entirely outside the app.
• Retrieval quality gates everything downstream — investing in RTS fallbacks paid off more than any prompt tuning.

What's next

More pipelines and richer persistence targets, proactive digests (surfacing decisions before someone has to ask), semantic-search rollout across workspaces, and a path to the Slack Marketplace so any team can install org memory in a click.

Built With

• ai-refinery
• anthropic
• block-kit
• confluence
• docker
• json-rpc
• mcp
• model-context-protocol
• node.js
• notion
• openai
• real-time-search-api
• slack-api
• slack-bolt
• slack-web-api
• socket-mode
• typescript
• zod