ScopeGuard

An AI Slack agent that watches client channels, compares requests against your Statement of Work, and privately alerts PMs when it detects scope drift, before it becomes scope creep.

Comment

ScopeGuard — AI-Powered Scope Drift Detection for Slack

How to Test ScopeGuard

Follow these steps in order to set up and test ScopeGuard from scratch:

Step 1: Upload a SOW Document

  • Open a DM with ScopeGuard
  • Type upload <project-name> and attach a SOW file (.txt, .md, or .pdf)
  • The bot will confirm the SOW has been saved to the database

Step 2: Link a Channel to the Project

  • Go to the channel you want to monitor
  • Run: /scopeguard setup <project-name> <project-name>.md
  • The bot will confirm the channel is now linked and being monitored

Step 3: Test Passive Monitoring

  • Passive monitoring only detects messages from users other than the PM (the person who ran /scopeguard setup)
  • Have a different user in the channel type a message like: Can we add an AI chatbot to the home page?
  • ScopeGuard will silently analyze the message and send an ephemeral alert (visible only to the PM) if it detects scope drift

Step 4: Test Manual Scope Check

  • Run: /scopeguard check Can we add a dark mode to the site?
  • The bot will return a verdict, reasoning, and a suggested response if the request is out of scope
  • Slash commands are completely invisible to other users in the channel

Step 5: Test @ScopeGuard Mention

  • Type: @ScopeGuard Can we integrate Stripe payments?
  • The bot will respond with a full scope analysis visible only to you
  • Note: Only the PM can use this feature. However, the mention message itself is visible to everyone in the channel. For discreet scope checks in client-facing channels, use /scopeguard check or the right-click shortcut instead

Step 6: View Project Info & Manage Mappings

  • /scopeguard status — Shows the project linked to the current channel
  • /scopeguard projects — Lists all configured projects
  • /scopeguard reassign <@user> — Transfer PM alerts to another user
  • /scopeguard remove — Unlink the project from the current channel
  • /scopeguard help — Shows all available commands

Inspiration

Scope creep is one of the most common reasons projects go over budget and miss deadlines. In agency and consulting environments, clients frequently make requests during casual Slack conversations that fall outside the agreed Statement of Work (SOW) — often without either side realizing it until invoicing time.

Project managers are expected to catch these moments in real time, but when you're juggling multiple channels, multiple clients, and dozens of daily messages, things slip through. By the time someone notices, the team has already spent hours on unapproved work.

ScopeGuard was built to solve this exact problem. It acts as a silent, always-on assistant that watches client channels, understands the SOW, and privately alerts the PM the moment a conversation drifts outside the agreed scope — before it turns into unpaid work.

How We Built It

ScopeGuard is built as a Slack agent using the Bolt for JavaScript framework with Socket Mode for real-time event handling. The architecture follows a multi-stage AI pipeline:

Core Pipeline

  1. Intent Classification — Every message in a monitored channel is analyzed by an LLM (Groq/LLaMA 3.1) to determine whether it contains a feature request, change request, or scope-relevant ask.
  2. Drift Analysis — Detected requests are compared against the project's SOW document. The AI evaluates whether the request falls in scope, out of scope, or is ambiguous, along with a confidence score and risk level.
  3. Change Request Drafting — For out-of-scope or ambiguous requests, the system generates a professional draft response the PM can copy, modify, and send to the client.
  4. Ephemeral Alerts — All alerts are delivered as ephemeral messages visible only to the PM. The client never sees the analysis happening.

Technology Stack

  • Slack Bolt (Socket Mode) — Real-time event handling for messages, commands, shortcuts, and mentions
  • Groq API (LLaMA 3.1 8B) — Fast LLM inference for intent classification, drift analysis, and CR drafting
  • MCP Server — Exposes SOW management, project mappings, and drift history as tools via the Model Context Protocol, enabling external AI agents to interact with ScopeGuard's data
  • PostgreSQL (Neon) — Cloud-hosted database for project mappings, SOW storage, and audit trail
  • Render — Cloud deployment for 24/7 availability

Key Features

  • /scopeguard setup <project-name> <sow-file> — Link a Slack channel to a project SOW
  • /scopeguard check <request> — Manually check a request against the SOW
  • /scopeguard status — Show the project mapped to the current channel
  • /scopeguard projects — List all configured projects
  • /scopeguard reassign <@user> — Transfer PM alerts to another user
  • /scopeguard remove — Unlink the project from the current channel
  • /scopeguard help — Show the help message with all available commands
  • upload <project-name> — Upload a SOW document (.txt, .md, or .pdf) via DM with a file attachment
  • @ScopeGuard <request> — Tag the bot with a request for instant scope analysis (PM only)
  • Right-click shortcut — PM can right-click any client message and select "Check Scope"
  • Passive monitoring — Automatically analyzes all messages in mapped channels and alerts the PM on scope drift
  • MCP server integration — External AI tools can fetch SOWs, query drift history, and list project mappings

What We Learned

  • Socket Mode vs. HTTP Mode — Socket Mode simplifies local development significantly (no need for ngrok or public URLs), but introduces challenges when deploying to cloud platforms that expect HTTP traffic. We solved this by running a standalone health check HTTP server alongside the Socket Mode connection.
  • Ephemeral messaging patterns — Designing a system where analysis happens transparently and results are shown only to the PM required careful thought about Slack's ephemeral message API and when to use it versus regular messages.
  • LLM prompt engineering for structured output — Getting consistent, parseable JSON responses from the LLM required iterative prompt refinement. The difference between a vague prompt and a well-structured one was the difference between 50% and 95% accuracy in scope classification.
  • Stateless cloud deployment — Moving from SQLite (local file) to PostgreSQL (Neon) was essential for cloud deployment since platforms like Render don't persist local files across restarts.

Challenges We Faced

  1. File upload handling in Slack — Slack's file upload events have a file_share subtype that Bolt's default message handlers filter out. Debugging why uploaded SOW files weren't being processed required deep investigation into Bolt's middleware chain and subtype filtering logic.

  2. Database migration under constraints — The initial SQLite setup worked perfectly for local development, but cloud deployment required a database that persists independently. We evaluated Render's managed PostgreSQL (30-day expiry on free tier), Supabase (pauses after 7 days of inactivity), and ultimately chose Neon for its automatic scale-to-zero with instant wake-up and no expiry on the free tier.

  3. Keeping the bot alive on free hosting — Render's free tier spins down after 15 minutes of inactivity. Since our bot relies on a persistent WebSocket connection, any sleep would disconnect it from Slack entirely. We solved this with an UptimeRobot monitor that pings a /healthz endpoint every 5 minutes.

  4. Balancing sensitivity — Tuning the intent classifier to avoid false positives (alerting on casual conversation) while catching genuine scope drift required careful confidence thresholds. We settled on a 60% confidence minimum to reduce noise while maintaining coverage.

Built With

Share this project:

Updates

Submission history