- Connect the Slack MCP server to give the agent access to channel history and search, so it can pull context from recent conversations when reviewing a feature
Inspiration
I'm a Senior QA Engineer with over a decade of experience testing hardware and software products. Lately, more code in my day-to-day work is AI-generated or "vibe-coded" — written quickly with AI assistance and merged without the scrutiny a human-written PR would normally get. I'd already started documenting common failure patterns in a public QA handbook called vibe-qa. This hackathon was the perfect excuse to turn that handbook into something interactive: a Slack agent that runs the checklist automatically wherever engineers are already talking.
What it does
Vibe-QA Agent reviews AI-generated code or feature descriptions against a 7-category QA checklist:
- AI hallucination risk
- Missing edge case coverage
- Hardcoded or brittle values
- Inadequate error handling
- Test coverage gaps
- Security surface
- AI-generated boilerplate risk
You can DM the agent directly, @mention it in a channel, or use the /vibe-qa slash command. It replies with a structured risk level (Critical/High/Medium/Low), specific flags tied to each checklist category, and a recommended test focus so a QA engineer knows exactly where to start.
How I built it
The agent is built on Slack's AI assistant platform using Bolt for Python. It uses the Slack assistant thread API — specifically the AssistantThreadStarted event, say_stream for streaming responses, and set_status with loading messages — which are Slack AI capabilities designed specifically for building intelligent agents inside Slack. The Slack CLI scaffolded the project from the Bolt for Python Starter Agent template, which also includes the Slack MCP server configuration (is_mcp_enabled is set in manifest.json), laying the groundwork for future MCP-based tool integrations like fetching PR diffs directly from GitHub.
I wrote a system prompt encoding the vibe-qa checklist, then wired up a /vibe-qa slash command alongside the existing DM and @mention flows. The AI backend runs on Groq's free tier (Llama 3.3 70B), chosen so the project has zero ongoing cost — anyone can fork it and run it without a paid API key.
Challenges I faced
The biggest challenge was formatting. Slack's mrkdwn format differs from standard markdown — double asterisks for bold and dash-prefixed bullets don't render correctly. The model's output used standard markdown, so I wrote a small conversion function to translate **bold** to Slack's *bold* and convert bullet markers properly before sending responses through the slash command handler.
I also worked through Slack's sandbox provisioning and OAuth scoping for the first time, since this was my first project using the Slack API and developer platform.
What's next
- Fetch the checklist live from the GitHub repo via the GitHub MCP server, so updates to the handbook automatically reflect in the agent
- Fetch PR diffs directly from GitHub URLs instead of requiring pasted code
- Add a feedback loop so flagged issues can be marked as false positives to refine future responses
Log in or sign up for Devpost to join the conversation.