Inspiration
When a microservice crashes in production, developers waste hours trapped in a disjointed workflow: stitching together raw terminal tracebacks, writing manual queries in Splunk to check for recurrence, and hunting through massive codebases to find the root cause. We realized that standard LLM tools lack the historical context of a system, and standard logging tools lack the intelligence to write code. We built Infer RCA to bridge this gap, transforming Splunk from a passive log vault into an active, intelligent memory bank for automated Root Cause Analysis.
What it does
Infer is an observability tool that separates application execution from error diagnostics.
Passive Interception: A non-blocking FastAPI middleware catches unhandled exceptions in the target app, hashes the involved source code, and vaults a lightweight event payload to a local Splunk database via the HTTP Event Collector (HEC).
Visual Mapping: It utilizes Python's native AST parser to dynamically map the microservice architecture in a React Flow dashboard, instantly highlighting failing nodes.
Contextual Diagnosis: When a user requests a diagnosis, the agent uses the Splunk Model Context Protocol (MCP) Server to query historical trends for that specific error. It then feeds the Splunk history, the traceback, and the local source code into Groq (Llama-3.3-70b) to generate a precise, source-level Markdown fix.
How we built it
Infer is built across three distinct layers:
The Client Middleware (FastAPI/Starlette): Runs inside the target application to intercept errors and extract tracebacks.
The Local Hub (Python/Uvicorn): A standalone ingestion server that manages local file state, orchestrates background tasks to prevent target-app blocking, and handles the Splunk HEC communication.
The Intelligence Layer: Integrates the Splunk MCP Server, a ChromaDB vector database for semantic caching of previously solved errors (via Google Generative AI Embeddings), and the Groq inference engine for zero-latency code generation.
The Frontend: Built with React, React Flow, and Tailwind CSS to provide a live log trail and an interactive architecture map.
Challenges we ran into
- Network Bloat & The Payload Problem: Sending entire source code files via HTTP on every single error completely bottlenecked the network. We engineered a Content-Addressed File Caching system. The middleware now generates SHA-256 hashes of all files in the stack trace. The ingestion hub checks its local cache and only requests file uploads for unrecognized hashes.
- The Timeout Race Condition: Initially, our agent was writing logs to Splunk synchronously, causing the target application to time out before it could upload its missing code. We resolved this by migrating all Splunk HEC vaulting to FastAPI BackgroundTasks, reducing our agent's response time to milliseconds.
- Traceback Volatility in Vector Search: We implemented ChromaDB to cache AI responses and save API tokens. However, exact identical errors were generating different vector embeddings because tracebacks include volatile memory addresses (e.g., 0x0001FA). We built a custom regex sanitizer to strip these memory allocations before embedding, drastically improving our semantic cache hit rate.
Accomplishments that we're proud of
- Zero-Regex Architecture Mapping: We successfully bypassed brittle regex parsing for our frontend. By utilizing Python's native ast library to walk the Abstract Syntax Tree for Import and ImportFrom nodes, we generate a highly accurate, deterministic microservice graph for the UI.
- Two-Way Splunk Integration: We didn't just dump logs into Splunk. By integrating the Splunk MCP Server into the diagnostic pipeline, the AI actively "reads" from the database to determine if a bug is a first-time anomaly or a systemic issue, drastically improving the quality of the generated fix.
What we learned
We gained deep, practical experience in distributed system race conditions, the complexities of building non-blocking ASGI middleware, and the immense value of the Model Context Protocol (MCP) for grounding Large Language Models in verifiable, historical operational data.
What's next for Infer RCA
- LRU Cache Eviction: Implementing a Least Recently Used chronological eviction strategy for the local file cache to manage disk space for massive enterprise codebases.
- Cross-Language AST Parsing: Expanding the architecture mapper to support polyglot environments, seamlessly tracking errors across a Node.js API gateway to a Python backend.
- IDE Integration: Bringing the Infer dashboard directly into VS Code extensions so developers never have to leave their editor.
Log in or sign up for Devpost to join the conversation.