AiOps-IntelligenceHub

Inspiration

The rapid adoption of Large Language Models (LLMs) in enterprise applications has created a critical blind spot: data leakage. Developers often inadvertently send PII (emails, SSNs), secrets (API keys, tokens), and proprietary code to LLMs via prompts. Traditional observability tools like Splunk excel at storing logs but struggle to proactively analyze unstructured prompt data in real-time or mask sensitive information before it leaves the network. We were inspired to build a bridge between proactive AI security and reactive enterprise observability.

What We Learned

Building AIOps Navigator taught us the complexities of local-first architecture. We learned how to balance low-latency regex detection with the nuanced understanding of LLM-based classification without incurring high costs or latency. Integrating deeply with Splunk's HEC (HTTP Event Collector) and Search API revealed the power of bidirectional data flow: pushing masked events for compliance while pulling aggregated analytics for dashboards. We also mastered Server-Sent Events (SSE) in Bun to stream replay simulations smoothly to the frontend.

How We Built It

We built AIOps Navigator as a full-stack TypeScript application using:

• Runtime: Bun for high-performance server-side rendering and API handling.
• Framework: TanStack Start (React 19) for a type-safe, modern UI.
• Database & Auth: Supabase for managing organization rules, user sessions, and replay datasets.
• AI Integration: OpenRouter API to dynamically select models (GPT-4, Claude, etc.) for deep semantic analysis of prompts.
• Observability: Native integration with Splunk via HEC for event streaming and REST API for historical trend analysis.

The core logic resides in src/lib/guardrail.server.ts, where we implemented a hybrid detection engine:

1. Deterministic Layer: High-speed Regex patterns for known entities (SSN, Credit Cards, AWS Keys).
2. Probabilistic Layer: LLM context analysis for custom organizational keywords and subtle data leaks.

Challenges We Faced

• Latency vs. Accuracy: Running an LLM check on every prompt adds latency. We solved this by making LLM analysis optional and configurable, defaulting to fast regex for common PII.
• Data Privacy in Logs: Sending raw prompts to Splunk violates compliance. We engineered a masking pipeline that redacts sensitive data before it is serialized to the HEC payload, ensuring Splunk only receives [REDACTED] placeholders.
• Local Development Experience: Ensuring the app works seamlessly with a local Splunk developer license required robust error handling for network timeouts and self-signed certificates.

Built With

• bun
• docker
• openrouter-api
• server-sent-events-(sse)
• shadcn
• splunk-(hec-&-search-api)
• supabase
• tailwind-css
• tanstack-start-(react-19)
• typescript