InsightWeaver

https://github.com/Zenieverse/InsightWeaver Zenieverse @ gmail dot com 💡 Inspiration Teams are drowning in unstructured data — chat logs, support tickets, PDFs, meeting notes — and the real insights often get lost. We wanted to build an autonomous assistant that doesn’t just search, but analyzes, summarizes, and acts. With TiDB Serverless and its new vector search, we saw a chance to build something that turns raw text into immediate, actionable intelligence. 🚀 What it does InsightWeaver Agent ingests documents (PDFs, logs, transcripts), indexes them in TiDB Serverless with vector embeddings, and allows hybrid search (semantic + keyword). It then chains in an LLM to summarize results and triggers external actions (Slack alerts or Trello tasks) automatically. Example: Upload customer support logs → query “What are the top refund issues?” → InsightWeaver clusters complaints → posts urgent findings to Slack → creates follow-up tasks in Trello. 🛠 How we built it Data Ingestion: Extracted text from uploads, generated embeddings, stored vectors + metadata in TiDB Serverless. Search: Implemented hybrid vector + full-text retrieval with TiDB Cloud’s vector search. LLM Orchestration: Chained search results into GPT-4 for summarization and recommendations. External Actions: Integrated with Slack Webhooks for real-time alerts and Trello API for task creation. Workflow: Built the entire pipeline in an App Canvas flow, ensuring smooth multi-step execution. ⚡ Challenges we ran into Schema design: Balancing embeddings with metadata fields for efficient retrieval. Vector search tuning: Adjusting similarity thresholds so semantic results were relevant without drowning in noise. Multi-API orchestration: Ensuring smooth handoff between TiDB → LLM → Slack/Trello. Time constraints: Packaging everything in a way that judges could run with minimal setup. 🏆 Accomplishments that we're proud of Successfully demonstrated end-to-end agentic automation powered by TiDB vector search. Built a reusable workflow template that can be adapted to customer support, compliance monitoring, meeting summarization, and more. Created a hackathon-ready open source repo with clear docs, demo dataset, and run instructions. 📚 What we learned How TiDB Serverless seamlessly combines transactional queries with vector search, making it a natural fit for retrieval-augmented generation (RAG). The importance of hybrid retrieval — vector search alone misses keywords, while keywords alone miss semantics. Together, they shine. Practical lessons on chaining LLM calls + external APIs in real workflows. 🔮 What’s next for InsightWeaver More integrations: Expand to Jira, Notion, Google Drive, and email. Real-time pipelines: Stream data ingestion from live chat or IoT sensors. Analytics dashboards: Add visualization of trends (refund categories, issue clusters). Team collaboration features: Assign tasks automatically to team members. Scaling: Deploy as a SaaS for enterprises who need intelligent insight automation.

Built With

• all