Chronos SRE Agent

Home page

Inspiration AI agents are rapidly transitioning from experimental scripts to core production workflows—writing code, operating deployment pipelines, and managing database clusters. However, in mission-critical environments like Site Reliability Engineering (SRE), agents face a fatal flaw: volatile memory.

Traditional databases are optimized for human-scale reads and writes, but autonomous agentic systems spawn dynamically, write constantly, and fail catastrophically if their memory goes offline. An agent that loses context or resets during an active outage is a liability.

Our inspiration was to build a self-healing AI SRE engineer that relies on a globally distributed, always-on system of record. We built Chronos SRE Agent—an agent whose memory persists across regions, failures, and scale, ensuring that learned remediation paths are never lost.

What it does Chronos SRE Agent is an autonomous, event-driven Site Reliability Engineer that detects cloud outages, diagnoses failures, triggers database recoveries, and persists postmortem findings.

Key features include:

Multi-Region Consensus Failover Map: Detects region-wide outages (e.g. AWS N. Virginia failure) and coordinates traffic rerouting while surviving CockroachDB replicas preserve global consensus. Self-Healing Vector Memory: Searches past incident records using pgvector semantic query operators. If it recalls a similar incident, it bypasses full LLM reasoning paths and triggers the fix instantly. Automated S3 Backups: Provisions cluster snapshots using the ccloud CLI and stores them in Amazon S3 prior to executing critical repair routines. Double-Run Learning Loop: The second time an identical incident occurs, Chronos recalls its previous postmortem and applies the resolution in under 3 seconds instead of running full diagnostic loops. How we built it We developed Chronos SRE Agent as a serverless microservices framework integrated with a high-fidelity observability dashboard:

AI Orchestrator: Built using TypeScript and hosted on AWS Lambda, leveraging Amazon Bedrock (Claude 3.5 Sonnet) as the core decision planning engine. Persistent Memory Layer: Utilized CockroachDB Cloud as the central pgvector store. We evaluate query similarity against historical postmortem embeddings using: Cosine Similarity If the match score satisfies our threshold: Similarity

0.85 Similarity>0.85 Chronos applies the cached remediation workflow directly. Observability Console: Built a React-Vite dashboard featuring an Onyx Black & Deep Royal Blue theme. It visualizes animated SVG node connections, latency heatmaps, active S3 file explorers, RAG prompt assemblers, and a query SQL playground. Challenges we ran into Our biggest challenge was designing the Double-Run Memory Feedback Loop. Creating a system that dynamically switches from slow reasoning (evaluating metrics, calling Bedrock, scanning runbooks) to fast recall (instantly applying the cached CockroachDB vector solution) required tight timing synchronization between database inserts and agent step callbacks. We resolved this by building a precise client-side SQL evaluator that indexes postmortems dynamically and calculates similarity vectors in real-time.

Accomplishments that we're proud of We are proud of building a platform that feels like a production tool (similar to Datadog or Grafana) rather than a simple monitoring dashboard. We created a live, interactive SRE experience where technical judges can trigger complex failures, view the active prompt engineering, browse S3 backups, and execute SQL statements against the vector database schema directly in the browser.

What we learned We learned that building reliable agentic architectures requires transitioning away from local vector caches toward globally consistent databases. CockroachDB's distributed quorum guarantees that even if a major cloud provider region goes completely offline, our agentic memory layer remains online, consistent, and readable, making it the perfect foundation for enterprise-ready AI.

What's next for Chronos SRE Agent Next, we want to connect Chronos to live Kubernetes API hooks and active Amazon CloudWatch metrics to resolve actual cloud outages. We also plan to scale the consensus map from 3 to 10+ regions, and integrate multi-agent collaborative memory routing, allowing different SRE containers to share incident learnings globally in real-time.