Workspace Historian

A Slack agent that turns channel conversations into a searchable decision knowledge base - AI drafts every entry, a human curator must approve it, and raw messages are never stored.

Comment

Workspace Historian - Project Story

Inspiration

Every team has that moment: someone asks, "Wait, why did we choose Postgres over Mongo?" or "Didn't we already decide not to do that?" The answer gets lost six weeks back in a Slack channel, buried between lunch plans and GIF exchanges. The decision was made openly, through genuine discussion, but it vanished as soon as the channel scrolled past.

We wanted to create something that behaves like a good notetaker during a meeting. It should quietly observe, recognize when a decision is made, and record it—without anyone needing to remember to save the important part of the conversation.

What it does

Workspace Historian is a Slack agent that:

  • Reads the history of any public channel it's invited to and continues listening afterward
  • Uses an LLM to identify the shape of a decision—not just any message, but the moment a team reaches a choice and the reasoning behind it
  • Drafts a structured entry (title, summary, reasoning, participants, source link) and sends it to a human curator for approval
  • Never auto-publishes. A person always has the final say before anything is added to the knowledge base
  • Allows anyone to browse or ask questions about published decisions—from the Home tab, a /historian command, or Slack's native AI agent panel—with answers linking back to the original discussion

How we built it

The architecture is intentionally straightforward in the middle and interesting at the edges. A Slack Bolt app in Node.js runs on Fly.io, using SQLite for storage. On top of that:

  • Classification runs through a swappable LLM layer (we support Groq, Gemini, and Anthropic interchangeably) so costs and latency can be adjusted without rewriting the pipeline
  • A lexical prefilter determines which message windows are worth an LLM call—decision-heavy language ("let's go with," "sounds good," "approved") gets classified; small talk does not. This helps control costs, but the model still makes the final decision.
  • Zero-copy handling: raw messages are pulled, processed in memory, and discarded. Only the structured decision—title, summary, participant IDs, a permalink—is saved. Slack data is never used to train a model.
  • A human review step exists between "AI thinks this was a decision" and "this is now in the knowledge base." Curators receive a DM (and, once fixed, a card in the agent chat thread) with Approve, Edit, or Reject buttons.

What we learned

The gap between "the demo works" and "real conversations work" turned out to be the main challenge of the project.

Our first version of live detection checked just the newest message for a decision keyword. During tests with a scripted exchange ("We need a database" → "I'd choose Postgres" → "Agreed" → "Let's go with Postgres then" → "Approved"), it worked well because the last message had a trigger word.

But real conversations don’t behave like that. A team member may close the decision with "Exactly," "Sounds good," or just "👍," with no keyword present, even though the decision was clear from the context two messages earlier. We had to redesign the live detection to check a rolling window of recent messages instead of only the latest one and add a safety mechanism to ensure that nothing could slip through an entire conversation without a trigger word. That single issue taught us more about how people actually communicate than any amount of pre-planning.

We also discovered that Slack's platform quirks can complicate things in specific ways. Enabling the AI agent experience replaces a workspace's traditional DM interface with a Chat/History split. This meant that when a curator tapped a mobile notification, it took them to the agent chat, while the actual Approve/Reject buttons were posted to a DM history tab they never accessed. The solution wasn’t to report a bug; it was a UX decision: we made sure to display the pending review queue as the first thing the agent shows when a curator opens the thread, so the action appears where the notification takes them.

Challenges we faced

  • The "one message, no signal" problem mentioned earlier—casual, decision-oriented language with no keyword overlap.
  • Where does the review UI actually appear? Slack's agent-enabled workspaces handle notifications differently than classic DMs. We had to design around platform behavior rather than fight against it.
  • Marketplace and domain infrastructure became an adventure of its own: our marketing site operated on one platform (Lovable Cloud) while the backend ran on another (Fly.io). Getting TLS to validate correctly involved tracing a redirect chain through DNS, finding a subdomain we didn’t control, purchasing a new domain, and ensuring every URL referenced infrastructure that could actually serve it. TLS termination must occur where DNS truly resolves, and no code changes can fix a domain issue.
  • Getting five independent installations verified for Marketplace review in the final 24 hours meant tracing the exact meaning of "active installation" by querying our installation store directly instead of relying on a dashboard that updates once a day.

None of these were the tough AI challenges we anticipated. They were the mundane details—timing, platform quirks, DNS—that determine whether a good idea actually reaches anyone.

Accomplishments that we're proud of

  • A review gate that's real, not decorative. It would have been easy to make "human approval" a rubber stamp — one button, no friction. Instead every proposed entry shows its full reasoning, participants, and source link before a curator commits, and Edit/Reject are as easy to reach as Approve. Nothing enters the knowledge base without a person actually looking at it.
  • Zero-copy held up under real use, not just in the privacy policy. It's one thing to promise "we don't store your messages" in a doc; it's another to build the pipeline so that raw text is structurally incapable of persisting — pulled, processed, discarded, every time. We're proud that this was true from day one rather than bolted on for compliance.
  • Catching the "casual decision" gap ourselves, before a user's trust broke on it. When live detection missed decisions phrased naturally instead of formally, we didn't patch around the symptom — we rebuilt the detection window and added a safety-net sweep so the failure mode can't recur silently. Shipping a fix that closes a whole class of bug, not just the one repro, is the kind of work we're proudest of.
  • Meeting a curator where they actually are. Realizing that Slack's own agent UI was hiding our review cards from mobile users, and fixing it by surfacing the queue directly in the chat thread rather than filing it as "a Slack limitation," felt like the difference between building a feature and building a product.
  • A swappable model layer that didn't slow anything down. Supporting Groq, Gemini, and Anthropic interchangeably behind one interface meant we could optimize for cost and latency without ever touching the classification or Q&A logic — infrastructure flexibility that stayed invisible to the user.
  • Shipping through real infrastructure headaches, not just clean local demos — chasing a TLS failure through DNS, discovering a subdomain we didn't own, migrating domains, and verifying installs against our own database when a dashboard lagged — and getting a genuinely deployed, publicly reachable app out the other side.

What's next for Workspace Historian

  • Thread-level and cross-channel decision linking. Right now each entry stands alone; the natural next step is detecting when a new decision revises or conflicts with an earlier one, and surfacing that history instead of letting contradictory entries sit side by side.
  • Confidence-aware curation. The classifier already scores its own confidence — surfacing that in the review card (and auto-deprioritizing low-confidence candidates in the queue) would let curators triage faster instead of reviewing everything with equal weight.
  • Private-channel support, opt-in and scoped. We deliberately limited the beta to public channels for privacy and simplicity; a carefully scoped, explicitly-invited private-channel mode is the most-requested extension we can already anticipate.
  • Richer answers via broader Real-Time Search use. We integrate Slack's RTS API for enrichment today; extending it to proactively pull related decisions when a new candidate is proposed (not just when a member asks a question) could help curators catch duplicate or conflicting decisions at review time, not after.
  • Team-level analytics. A lightweight "decisions per month," "most active decision-makers," or "topics we keep re-deciding" view would turn the knowledge base from a lookup tool into a mirror teams can use to improve how they actually make decisions.
  • Deeper Marketplace presence — once we're through review, expanding install support, formal SOC 2-style documentation, and enterprise-grid (org-wide) installs are the natural path from hackathon project to something teams can rely on long-term.

Built With

Share this project:

Updates