Inspiration
Every day, teams share massive amounts of valuable context in chat channels like Telegram. We paste reference links, write notes, share diagrams, and send voice memos. But this collective intelligence is lost almost immediately as the chat scrolls off the screen. Decisions get buried, links are forgotten, and context-dependent tasks are lost in the daily noise.
We wanted to build a "second brain" for teams—a system that actively listens, captures shared knowledge passively, and structures it into an actionable memory layer without requiring team members to manually copy, paste, or organize anything themselves.
What it does
Arnheid is an autonomous, context-aware AI agent that sits directly in your Telegram groups and DMs.
- Ambient Capture: When you post a link in a group chat, the bot passively scrapes the webpage, cleans the text, and processes it in the background. If you share a voice note, it transcribes the audio; if you share a photo, it describes the image.
- Unified Memory (RAG): All captured data is embedded and stored alongside relational metadata. When a user asks a question, Arnheid uses a hybrid search (vector search combined with keyword matching) to pull the exact context.
- Knowledge Graphs: It periodically extracts entities (people, tools, concepts) and their relationships (such as contradicts, related to, or follows up) from the data, building a structured network of your team's knowledge.
- Context Resolution: The agent buffers recent chat messages so that when you ask it a question, it understands pronouns or ambiguous terms (like "the meeting we discussed earlier").
- Autonomous Actions: Using its agentic reasoning loop and GSuite tool integrations (via the Model Context Protocol), the agent can draft emails, schedule calendar events, and search documents based on the conversation.
- Live Dashboard: A React dashboard monitors the database, showing cluster health, backup status, and counts of stored items, entities, edges, and vector chunks.
How we built it
We built Arnheid on a modern, highly reliable stack to ensure durability and scalability:
- Backend Compute: A single compiled Rust binary using teloxide for the Telegram interface, axum for HTTP webhook routing, and sqlx for database connection pooling. The entire application runs inside Docker containers on an AWS EC2 instance.
- Storage & Memory Layer: We chose CockroachDB Cloud Serverless as our unified data store. Rather than syncing a relational database and a separate vector database, we consolidated everything. Metadata, chat buffers, knowledge graph nodes, and high-dimensional semantic vectors (1024 dimensions) are stored in the same database.
- Background Processing: We built a durable ingestion queue inside CockroachDB (ingestion_jobs). A Rust worker polls this table and executes summarization, transcription, and embedding tasks asynchronously with exponential backoff retries.
- LLM Orchestration: The bot runs a 5-turn reasoning loop using Hugging Face for generating text embeddings and Anthropic Claude models for summarization, intent routing, and tool execution.
- Security & Network: The EC2 instance connects to CockroachDB Cloud via WebPKI-verified SSL/TLS. We also integrated the CockroachDB Cloud Managed MCP server to allow the AI to safely query database tables during interactions.
Challenges we ran into
- Preventing Data Loss on Crashes: When processing media files or web scraping, external APIs sometimes timeout or fail. Initially, an application crash meant losing the job. We solved this by hosting our asynchronous queue directly inside CockroachDB. Using distributed transactional locks, we ensure that if a worker crashes mid-job, the task is automatically reclaimed and retried.
- Pathing and OS Differences: Building a cross-platform tool that manipulates local file directories for user workspaces caused file-system issues when shifting from local development (Windows) to live servers (Linux). We had to implement strict directory guards and handle empty paths carefully during recursive copies.
- Merging Vector and Relational Data: Combining keyword matches with vector similarity scores is notoriously complex. We solved this by implementing Reciprocal Rank Fusion (RRF) directly in SQL queries, joining relational tables and native VECTOR database columns in a single, low-latency transaction.
Accomplishments that we're proud of
- Zero Sync Overhead: We successfully eliminated the need for a separate vector database. CockroachDB native vector operations handle all of our semantic lookups alongside our relational knowledge graph.
- Highly Efficient Compute: The backend compiles down to a single lightweight Rust binary that handles webhook parsing, LLM routing, cron tasks, and Prometheus metric generation in-process, utilizing minimal CPU and RAM on a small EC2 instance.
- Reliable Operations: We created a background CLI monitoring task that runs hourly. It logs into Cockroach Cloud using ccloud, confirms the database cluster status is active, checks that daily backups are fresh, and notifies us via Gmail if anything requires attention.
What we learned
- Distributed Vector Databases: We learned how to write, index, and query multidimensional embeddings directly within SQL schemas, using C-SPANN indexes to achieve low-latency lookups.
- Durable Systems Design: We gained experience structuring application state such that stopping, starting, or redeploying the EC2 compute layer has zero impact on long-running worker tasks or active user conversations.
- Agentic Reasoning Loops: We learned how to build a robust JSON-parsed tool-calling parser that executes background GSuite actions safely without letting the LLM execute arbitrary commands outside its sandbox.
What's next for Arnheid
- Omnichannel Ingress: We plan to expand the bot's webhooks from Telegram to Slack, Discord, and Microsoft Teams.
- Extended MCP Tooling: We want to add real-time Google Docs editing, sandboxed code execution, and multi-engine web search verification (cross-checking facts across Google, Bing, and Tavily).
- 3D Visualizer: We are developing an interactive WebGL-powered 3D visualization on the dashboard that lets users visually map and browse the semantic connections in their knowledge graph.
Log in or sign up for Devpost to join the conversation.