Meeting Debt Collector

• 

  architecture diagram

• 

  commitment lifecycle diagram

• 

  rts check - update in any public channel tracked by agent - can ask for update in dm

• 

  can make multi commitments in 1 message

• 

  check prs against github

• 

  check prs against github - 2

• 

  can check prs against any public repo

• 

  cant delegate work for you

Inspiration

Every team has the same ghost haunting its Slack workspace: "I'll ship the PR by Friday," "I'll send that to legal by 5," "I'll get you the report tomorrow." These promises scroll off the screen and die. Nobody tracks them, nobody follows up, and by the next standup everyone's re-asking "did that ever happen?" We called this pile of unpaid promises meeting debt — and decided to build the bot that collects it.

We didn't want another reminders bot that just re-posts what someone said an hour later. We wanted something that checks ground truth — did the PR actually merge? — before it ever bothers a human.

What it does

Meeting Debt Collector is a Slack agent that:

1. Auto-captures commitments from every DM and @mention it sees. An LLM extractor (few-shot, JSON-mode, temp 0) reads the message plus thread context and decides if it's a genuine, checkable commitment — rejecting hedges ("I might…"), questions, sarcasm, past tense, and vague team-speak ("we should really…"). What survives gets persisted with an owner, a deadline, and any external reference (PR number, ticket, doc).
2. Verifies against reality, not vibes. For GitHub-linked commitments, it calls the real GitHub MCP server to check whether the PR is actually open, closed, or merged. For everything else, it queries Slack's Real-Time Search API to see if the commitment was already reported done elsewhere in the workspace — scoped to the user's own token, never the bot's.
3. Collects the debt weekly. A scheduled job (plus an on-demand "send me my digest" command) builds a Block Kit digest of what's still overdue. Verified-done items are silently dropped, still-open items show their real GitHub status, and anything unverifiable is flagged "check manually" instead of guessed at.

How we built it

Three technologies, three distinct jobs, deliberately kept from blurring together:

The app is built on Bolt for Python + Pydantic AI, scaffolded from Slack's own bolt-python-starter-agent template and running over Socket Mode. Each capability is a discrete tool the agent can call: extract_commitments, check_pr_status (GitHub MCP), check_rts_status (RTS), and send_weekly_digest_now (Block Kit composer). Commitments live in SQLite; a weekly scheduler wakes up, re-verifies every open item, and DMs each user their own digest.

We treated the extractor prompt as a first-class artifact, not an afterthought — it's built around a strict definition of "commitment" (first-person, concrete, deadline-bound) and validated against a 17-case suite covering the failure modes that break naive extractors: group intentions, sarcasm, past-tense reports, and commitments with no real deadline.

Challenges we ran into

• Keeping RTS and MCP from blurring together. It's tempting to reach for Slack search to answer "is the PR merged?" — but RTS never leaves Slack. Any external ground-truth check has to go through MCP, full stop.
• The extractor's untrusted deadline field. The model doesn't know today's date, so deadline_resolved coming out of the LLM can't be trusted directly — it has to be reconciled against wall-clock time downstream, not taken at face value.
• Hidden reasoning tokens. gpt-oss-120b burns tokens on reasoning before it ever writes output — a max_tokens that looks generous for a JSON blob can silently return an empty string if it's too tight.
• Local dev has no user token. Socket Mode dev sessions don't carry a per-request user token, so RTS hits its "no token" path locally by design — real user tokens only show up after an OAuth install, which shaped how we tested end to end.

Accomplishments that we're proud of

A bot that doesn't just parrot back what you said in Slack — it goes and checks. The three-technology pipeline (extract → verify internally → verify externally) means by the time a commitment shows up in your digest, it's already been checked against two independent sources of truth, not just re-surfaced from memory.

What we learned

That the hardest part of "AI reads your Slack and tells you what's true" isn't the LLM call — it's deciding what the LLM is not allowed to be trusted for (dates, "is this really done") and routing those questions to a system that actually knows the answer.

What's next for Meeting Debt Collector

Jira as a second ground-truth source alongside GitHub, a threshold for auto-escalating chronically overdue commitments to a channel instead of just a DM, and an Assistant-panel view so a commitment's verification status is visible without waiting for the weekly digest.

Built With

• bolt-framework
• github-api
• github-copilot
• groq
• llm
• mcp
• pydantic-ai
• python
• slack-api
• slack-bolt
• socket-mode
• sqlite