Community Beacon

An AI Slack agent that remembers every call for help — detects needs and offers of help in conversation, matches them to each other, and proactively escalates so nothing falls through the cracks.

Comment

Inspiration

Mutual-aid and nonprofit coordination happens almost entirely in Slack (or Slack-like tools), but it's structurally fragile: volunteers burn out re-reading threads to find who still needs help, and people in genuine need go unnoticed because their message didn't sound urgent — it was just a quiet, easy-to-scroll-past request. Existing tools treat this as a message-classification problem: label it and move on. We wanted to build something that behaves like an actual coordinator who's been in the channel the whole time and remembers everything — not "here's a message that might be a need," but "this is the fourth transport request from this channel in nine days, and here's a volunteer with five completed matches nearby."

The project's own build history reflects that ambition: it started as a general community help-request tracker, pivoted mid-hackathon to a B2B growth-intelligence agent to explore the platform's range, then pivoted back to the community-impact domain — this time built on the stronger architecture (MCP server, Real-Time Search + fallback, typed JS, unit tests, provider abstraction) that detour produced.

What it does

Community Beacon watches Slack channels it's invited to and, for every qualifying message, runs a full agentic reasoning loop:

  1. Detects 15 signal types across three categories — needs (help request, urgent need, transport, food insecurity, housing, medical, emotional support, resource request), offers (volunteer, donation, skill, resource available), and coordination (event coordination, gratitude, follow-up needed) — each with a confidence score, quoted evidence, AI reasoning, and a recommended action.
  2. Searches workspace history before deciding anything: a live Real-Time Search sweep combined with a persisted signal-memory aggregation (has this person asked before? how many times has this channel seen this need this month? is anyone's identical request still open? which volunteers have a track record here?), cached 60 seconds so repeat lookups are free.
  3. Summarizes with reasoning, not just facts: one LLM call over the message and the history object produces a recurrence summary, risk assessment, volunteer recommendation, confidence score, and an escalation recommendation.
  4. Matches needs to offers deterministically (type affinity, text similarity, volunteer track record, channel proximity, priority, historical success rate), then branches by confidence: HIGH → one-click Confirm, MEDIUM → coordinator Approve/Reject, LOW → posts outreach to a volunteers channel instead of guessing.
  5. Escalates proactively — an hourly sweep finds unresolved signals past a per-tier age threshold (critical/high/routine each have their own SLA), respects quiet hours and a max-reminders cap, and DMs coordinators with an AI-written explanation of why this needs attention now.
  6. Tracks a full reasoning timeline per signal (detected → history searched → context enriched → match decided → escalated → resolved), visible via a "View Timeline" button.
  7. Reports real impact — an App Home dashboard and /cb-impact surface time-to-match, response times by priority tier, auto-triage counts, volunteer utilization, a per-channel demand heatmap, and an estimated coordinator-hours-saved figure (explicitly labeled as a documented heuristic, not a precise measurement).
  8. Exposes the same pipeline as an MCP server (17 tools, including signal detection, priority scoring, matching, and 9 workspace-history/analytics tools like get_repeat_requesters and summarize_workspace_context) so any MCP client, not just Slack, can drive the community-intelligence engine.

Everything happens inside Slack: Block Kit alert cards with reasoning and action buttons, a native App Home dashboard, three slash commands (/cb-scan, /cb-needs, /cb-impact), and an AI assistant side-panel for direct Q&A.

How we built it

  • Platform: @slack/bolt v4 in Socket Mode (outbound WebSocket only — zero HTTP surface, no exposed ports, works behind any firewall).
  • Detection & reasoning: an LLM-based intent engine gated by a cheap keyword pre-filter and a per-channel rate limiter, so ordinary chatter never reaches the model. Default LLM backend is Hugging Face's free Inference Router (Llama 3.1 8B Instruct) via an OpenAI-compatible client, with local Ollama as a zero-cost alternative — same code path either way.
  • Workspace memory: fuses a live Real-Time Search sweep with structured aggregation over every signal ever detected, cached 60 seconds. When no fresh search token is available (e.g. the separate MCP process), it falls back to channel-history search plus keyword filtering — no hard failure either way.
  • Decision layers are deterministic and auditable, not LLM guesses: priority scoring (0–100, critical/high/routine) and match-confidence branching (HIGH/MEDIUM/LOW) are both weighted-factor formulas, not a model call — every factor is visible in the card's explanation text. The LLM's own confidence score in the coordinator summary is a separate, complementary signal.
  • Two independent processes, one service layer: the Bolt app and a separate MCP stdio server both import the same service modules and communicate only through on-disk JSON stores — no RPC between them, so a signal logged via an MCP tool call is immediately visible in Slack and vice versa.
  • Persistence: file-backed JSON stores with atomic writes and merge-on-save by signal ID (newest update wins) so the two processes writing concurrently don't clobber each other.
  • Case-log abstraction: one provider-lookup function is the only import path business logic uses; a fully functional mock provider ships by default, with HubSpot and Salesforce Nonprofit Cloud stubs ready for real credentials — swapping providers is a one-line env change, not a refactor.
  • Dashboard: rendered natively into Slack's App Home tab (Block Kit) — deliberately not a separate web server, to keep the whole system inside Slack's trust boundary and Socket Mode's zero-inbound-port model.
  • Typing & tests: incremental TypeScript checking (no rewrite) plus unit tests for every pure-logic service — 96 tests passing, tsc --noEmit clean.

Challenges we ran into

  • Search token availability: Slack's Real-Time Search requires a short-lived token that only arrives on live event payloads, never on slash-command payloads — and never at all in a separate MCP stdio process. Solved with an opportunistic token cache plus a channel-history fallback path, so the same search code works whether or not a fresh token exists.
  • Prompt injection via message text: a Slack message containing a literal triple-quote could prematurely close a prompt's fence and inject fake instructions after it. Fixed with a sanitization step that neutralizes the delimiter before any LLM call.
  • Silent data loss under two concurrent processes: the Bolt app and MCP server both write to the same signal store; the naive "load once, overwrite on save" pattern meant whichever process saved last discarded the other's writes since its last load — caught live when a detected signal vanished mid-test. Fixed with re-read-and-merge-by-signal-ID on every save.
  • A single emergency signal couldn't reach "critical" priority: the original weighting required two corroborating signal types to cross the critical threshold, so a lone "someone collapsed and needs medical help now" message scored only "high" and got a 4-hour escalation window instead of a 1-hour one. Found during live-workspace verification and fixed by re-weighting medical/urgent signals so one high-confidence signal alone is enough.
  • Escalation sweep could "fire" with nobody actually notified: if no coordinator DM list or alerts channel was configured, the sweep still marked signals as escalated and burned through the reminder cap — now it skips entirely rather than silently consuming a real signal's limited escalation attempts.
  • Test suite was destructive against real data: several tests wiped the production data files directly to reset state, which destroyed real accumulated demo data when run against a live-feeling workspace. Fixed by making store paths overridable via environment variables, with every test now pointed at an isolated temp file.

Accomplishments that we're proud of

  • A genuinely agentic loop — not "classify and post" but detect → search live + persisted history → reason over both together → decide a match with explainable confidence branching → escalate proactively → track a full timeline — verified end-to-end against a real Slack workspace, not just unit-tested in isolation.
  • 96/96 tests passing, clean TypeScript check, across the whole repo, with every defect found during live verification fixed and regression-tested rather than left as a known issue.
  • A security posture that was actually exercised: prompt-fence injection mitigation, rate-limiting scope, escalation-notification integrity, channel_types: ['public_channel'] hardcoded on every workspace search (private channels and DMs are never searched by RTS or the MCP workspace-history tools), and Zod-validated input schemas on all 17 MCP tools.
  • Zero HTTP surface — the entire system, including its analytics dashboard, runs over Socket Mode's outbound-only connection plus a stdio MCP process. Nothing to expose, nothing to firewall.
  • A case-log provider abstraction that makes "swap in real Salesforce Nonprofit Cloud credentials" a two-file change instead of a rewrite, so the path from hackathon demo to a real nonprofit's CRM is already paved.

What we learned

  • Reasoning that's "impossible from a single message alone" only becomes real if the pipeline actually queries live history and a persisted structured record before the LLM ever sees the text — that ordering (search first, summarize second) is what turns a plausible-sounding sentence into a verifiably grounded one.
  • Deterministic scoring for anything a human will act on (priority, match confidence) is worth the extra code versus asking an LLM to score itself — it's auditable, reproducible, and every factor is explainable in the UI, which matters enormously for something coordinating real human need.
  • Running two processes against the same on-disk store surfaces real distributed-systems problems (lost updates, races) even at "one laptop, two terminals" scale — the fixes were small, but only because we went looking for them via actual live-workspace testing instead of trusting that unit tests covering each module in isolation were enough.

What's next for Community Beacon

  • Implement real Salesforce Nonprofit Cloud and HubSpot providers once sandbox credentials are available (the stub interface is already the exact shape the mock provider satisfies).
  • Move the signal store and mock case log off flat JSON files to SQLite to remove the single-instance limitation and support horizontal scaling.
  • Make matching semantic (embedding similarity) instead of type-affinity plus weighted factors, so a "ride to a food bank" offer ranks above a generic transport offer for a food-insecurity need specifically.
  • Add an integration/end-to-end test suite for the Slack listeners themselves (today only the pure-logic services are unit-tested).
  • Add a scheduled trigger for the impact report instead of manual-only reporting.

Built With

Share this project:

Updates