Scrumy

Action items or tasks
Meeting detail screen
Meeting list view
chrome_extension

Inspiration

The inspiration for ScrumBot came from experiencing the universal frustration of meeting fatigue and lost action items. We've all been in meetings where important decisions are made and tasks are assigned, only to have them forgotten or buried in email threads. The problem becomes exponential in remote teams where meetings happen across different platforms (Google Meet, Zoom, Teams) and tasks get scattered across various tools (Notion, Slack, ClickUp).

We realized that while AI has revolutionized many workflows, meeting management remained stuck in manual note-taking and post-meeting cleanup. We wanted to build something that could capture any meeting conversation and automatically turn it into actionable, distributed tasks without any manual intervention.

What it does

ScrumBot is an AI-powered meeting assistant that automatically captures, processes, and distributes action items from video conferences. Here's how it works:

Real-time Meeting Processing: A Chrome extension captures audio directly from Google Meet, Zoom, and Microsoft Teams calls. The audio is streamed via WebSocket to our backend where Whisper.cpp performs local transcription for privacy.

AI Task Extraction: Groq's LLM analyzes the transcript to identify action items, assignees, priorities, and due dates. The AI understands natural conversation context and can extract meaningful tasks from casual discussion.

TiDB Serverless Storage: All meeting data, transcripts, and extracted tasks are stored in TiDB Serverless, providing both real-time operations and analytical capabilities for meeting insights.

Cross-Platform Integration: Using the Model Context Protocol (MCP), tasks are automatically created in Notion databases, Slack notifications are sent to team members, and tasks sync to ClickUp with proper assignees and metadata.

AI Chatbot Assistant: An intelligent assistant powered by TiDB's vector store capabilities allows users to query meeting history, check task status, and get insights about team productivity using natural language.

How we built it

We designed ScrumBot as a distributed microservices system with clear separation of concerns:

Chrome Extension: Built with vanilla JavaScript to integrate directly with video conferencing platforms. Implements Web Audio API for real-time audio capture and WebSocket streaming for low-latency transmission.

Real-time Processing Backend: FastAPI server with WebSocket support handles audio streaming. Integrated Whisper.cpp for local transcription and built chunked processing pipeline for continuous audio streams.

AI Processing Layer: Leveraged Groq's API for lightning-fast task extraction. Designed sophisticated prompts for context understanding and implemented confidence scoring for extracted tasks.

TiDB Serverless Integration: Created normalized database schema for meetings, tasks, and transcripts. Implemented vector storage for the AI chatbot knowledge base and built analytics queries for meeting insights.

Integration Layer: Implemented Model Context Protocol connectors for Notion, Slack, and ClickUp. Built retry mechanisms and error handling for reliable cross-platform task distribution.

AI Chatbot: Developed using sentence transformers and TiDB vector store. Implemented RAG (Retrieval-Augmented Generation) for contextual meeting queries and natural language task management.

Challenges we ran into

Real-time Audio Processing Complexity: Streaming audio from Chrome extension while maintaining low latency and handling network interruptions required implementing chunked WebSocket streaming with automatic reconnection and sophisticated buffer management.

Cross-Platform Integration Reliability: Different APIs (Notion, Slack, ClickUp) have varying rate limits, authentication methods, and data formats. We solved this by building a unified integration layer with exponential backoff and graceful degradation.

AI Task Extraction Accuracy: Extracting meaningful tasks from natural conversation while handling speech recognition errors required a multi-stage AI pipeline with grammar correction, context analysis, and confidence scoring.

Chrome Extension Security: Accessing audio from video conferencing platforms while respecting security boundaries required careful content script injection with proper permission management and user consent flows.

Database Schema Design: Creating a schema that supports both transactional operations and analytical queries while maintaining performance was solved by leveraging TiDB Serverless's hybrid OLTP/OLAP capabilities.

Accomplishments that we're proud of

Seamless Real-time Pipeline: We built a complete end-to-end system that processes meeting audio in real-time with sub-second latency from capture to task creation.

Privacy-First Architecture: All audio transcription happens locally using Whisper.cpp, ensuring sensitive meeting content never leaves the user's machine unnecessarily.

TiDB Integration Excellence: We fully leveraged TiDB Serverless for both operational data storage and analytical workloads, including implementing a vector store for our AI chatbot.

Cross-Platform Automation: Successfully implemented the Model Context Protocol to automatically distribute tasks across multiple platforms (Notion, Slack, ClickUp) with proper formatting and metadata.

Production-Ready Deployment: Built comprehensive deployment scripts with PM2 process management and triple ngrok tunneling for easy demonstration and scaling.

AI-Powered Insights: Created an intelligent chatbot that can answer complex questions about meeting history and task status using natural language queries against TiDB data.

What we learned

TiDB Serverless Power: TiDB's hybrid OLTP/OLAP capabilities are perfect for applications needing both real-time operations and analytics. The MySQL compatibility made integration seamless while providing enterprise-scale performance.

WebSocket Streaming Optimization: Proper buffering and chunking strategies are crucial for real-time audio processing. We learned to balance latency with reliability through careful buffer management.

AI Integration Best Practices: Groq's API provides excellent speed-accuracy balance for real-time applications. We discovered that multi-stage prompting with confidence scoring significantly improves task extraction quality.

Chrome Extension Architecture: Content scripts can be surprisingly powerful when properly architected. We learned to work within security constraints while maintaining seamless user experience.

Microservices Complexity: Managing distributed systems requires comprehensive error handling, monitoring, and deployment automation. We built robust retry mechanisms and health checking throughout the pipeline.

What's next for Scrumy

Enterprise Scaling: Implement multi-tenant architecture with role-based access control and enterprise security features for large organizations.

Advanced Analytics: Build comprehensive meeting analytics dashboard using TiDB's analytical capabilities to provide insights on team productivity, meeting effectiveness, and task completion patterns.

Mobile Integration: Develop mobile companion app for task management and meeting insights on-the-go, leveraging TiDB's global distribution capabilities.

Enhanced AI Capabilities: Integrate advanced speaker identification, sentiment analysis, and meeting outcome prediction using TiDB's vector storage for improved context understanding.

Real-time Collaboration: Implement live meeting dashboards where participants can see tasks being extracted in real-time and collaborate on task refinement during meetings.

Global Deployment: Leverage TiDB's global distribution to deploy ScrumBot worldwide with region-specific compliance and data residency requirements.