Chimera

Self-organizing AI agent societies that investigate, challenge, and resolve production incidents in 37 seconds with no humans paged.

Comment

Inspiration

At 2am, a P0 hits. An engineer gets paged, wakes up groggy, spends 45 minutes reading logs, forming hypotheses, second-guessing themselves. Meanwhile, the business bleeds; the average cost of a production incident is $5,600 per minute.

The real problem isn't the incident. It's that human diagnosis is sequential, biased, and exhausting. One engineer, one hypothesis at a time.

We asked, What if the investigation itself were a multi-agent system? What if instead of one tired engineer. You deployed a society of AI specialists that challenged each other's conclusions before acting?

That question became CHIMERA.

What it does

CHIMERA is an autonomous incident response platform powered by self-organizing AI agents. societies. Given any production incident, CHIMERA:

  1. Synthesizes the optimal agent topology on the fly: no hardcoded roles, no templates. Qwen Max reads the incident and invents the right specialists for that specific problem.

  2. Spawns a specialist society's agents with hyper-specific expertise run parallel Investigations using real tools: SQL queries, health checks, log analysis, deployment diffs.

  3. Deploys adversarial agents every specialist gets a paired adversarial agent whose Their only job is to challenge their conclusions. No confirmation bias. No rubber-stamping.

  4. Reaches consensus via Nash-inspired conflict resolution weighing specialist confidence against adversarial challenge strength to produce a validated decision.

  5. Learns from every resolution; successful topologies and diagnostic patterns are stored in pgvector memory. The second time CHIMERA sees a similar incident class, it recalls the proven team instantly 72× faster than resynthesizing.

Key metrics:

  • $P_0$ incident resolved in 37 seconds (vs 45-minute industry average)
  • 85%+ average agent confidence after adversarial validation
  • 72× faster resolution on recalled incident classes
  • $0 human engineering time for known incident patterns

How we built it

Agent architecture

The core insight: agents are not pre-defined. The Meta Orchestrator (Qwen Max) receives an incident and immediately delegates to the Topology Synthesizer (Qwen Plus), which generates a JSON agent specification, roles, system prompts, and tools entirely at runtime.

Every specialist agent runs a full multi-turn tool execution loop:

$$\text{think}(input) \rightarrow \text{tool_calls} \rightarrow \text{execute} \rightarrow \text{observe} \rightarrow \text{think}(\ldots)$$

This continues until Qwen returns a plain-text conclusion. Tool calls include real integrations: PostgreSQL queries, HTTP health checks, log searches, deployment history.

Adversarial validation

Each specialist is paired with an adversarial counterpart that receives the specialist's full analysis and is prompted to find logical gaps, missing evidence, and alternative hypotheses. Only challenges with confidence $\geq 0.60$ are factored into consensus.

Consensus engine

The consensus engine receives all specialist results $S = {s_1, s_2, \ldots, s_n}$ and adversarial challenges $A = {a_1, a_2, \ldots, a_m}$ and computes:

$$\text{confidence} = \frac{\sum_{i} s_i.\text{conf}}{|S|} - \lambda \cdot \frac{|{a \in A : a.\text{conf} \geq 0.6}|}{|A|}$$

Where $\lambda$ is the adversarial penalty weight. If $\text{confidence} < 0.65$, a human A checkpoint is triggered before any action is taken.

Memory system

Successful resolutions are embedded using text-embedding-v3 (1024 dimensions) and stored in PostgreSQL with pgvector. Recall uses cosine similarity:

$$\text{similarity}(q, d) = \frac{q \cdot d}{|q| |d|} > 0.88$$

Recalled topologies have learned diagnostic patterns injected into specialist system prompts. making agents progressively smarter at each incident class.

Infrastructure

  • Backend: NestJS + TypeScript
  • Agent messaging: Redis Streams with consumer groups, DLQ, and pending retry
  • LLM: Qwen Max (orchestration), Qwen Plus (specialists), text-embedding-v3 (memory)
  • Memory: PostgreSQL 16 + pgvector (IVFFlat index)
  • Storage: Alibaba Cloud OSS (postmortem reports)
  • Frontend: Next.js + React Flow (live agent graph visualization)
  • Real-time: Socket.io WebSocket with event replay on reconnect

Challenges we ran into

  1. Tool execution loop correctness Qwen's API requires tool_call_id to be round-tripped back with tool results. Missing this caused silent failures on multi-turn tool chains. We had to carefully Preserve raw assistant messages with their tool_calls arrays in conversation history.

  2. Race condition in result collection If a specialist responded between dispatchTasks() returning and waitFor() registering its listener, the result was silently dropped, causing a 2-minute timeout every run. Fixed by registering the listener before checking the map, then immediately flushing any results that arrived in the window.

  3. Redis Streams vs pub/sub Initial implementation used pub/sub messages that were lost if an agent wasn't listening at the exact moment of publication. Migrated to Redis Streams with consumer groups, which gave us at-least-once delivery, pending acknowledgment, and a dead-letter queue.

  4. Context window management Long multi-turn agent conversations would hit token limits mid-investigation. Built a pruning strategy that always preserves the first message (incident context) and the last 12 history entries, dropping the middle.

  5. Dynamic topology → JSON reliability LLMs occasionally return malformed JSON for the topology spec. Built a 3-attempt retry loop with corrective prompting ("your previous response was invalid JSON; try again.") that recovered 100% of synthesis failures in testing.

Accomplishments that we're proud of

  • Zero hardcoded agent roles — the system invents the right team for every incident
  • Adversarial validation as a first-class concept—not just multiple agents, but Agents specifically designed to attack conclusions
  • Memory that compounds — the system demonstrably gets faster with every incident, with a measurable speedup that judges can verify in the demo
  • Production-grade engineering — circuit breakers, DLQ, connection pooling, concurrent incident handling, WebSocket reconnect with event replay
  • Full analytics — MTTR trends, cost savings in USD, risk velocity scoring, downloadable postmortem reports with executive summaries

What we learned

  • Adversarial agents dramatically improve decision quality — without them, specialists tend to over-confirm their first hypothesis. The adversarial loop surface is missing evidence that changes the final decision in ~30% of cases.

  • Dynamic topology synthesis is feasible — we were uncertain whether an LLM could reliably generate valid agent specifications. With good prompting and retry logic, It works consistently and produces genuinely creative specialist combinations.

  • Memory retrieval threshold matters enormously—cosine similarity of 0.88 was the sweet spot. Below 0.85, unrelated incident classes polluted recalls. Above 0.92, Legitimate recalls were missed.

  • Redis Streams are dramatically better than pub/sub for agent messaging—the The consumer group + ACK model gave us reliability we couldn't achieve with pub/sub regardless of how carefully we handled subscriptions.

What's next for CHIMERA

  • Real integrations — live connectors to Datadog, PagerDuty, Elasticsearch, and Kubernetes events so CHIMERA responds to real production signals

  • Agent specialization marketplace — pre-trained specialist agents for common domains (database, Kubernetes, payment systems, CDN) that organizations can Plug into the topology synthesizer

  • Multi-tenant SaaS — isolated agent societies per organization, with shared learning across anonymized incident patterns (federated memory)

  • Causal graph construction — after resolution, CHIMERA builds a causal graph of the incident chain that feeds into proactive risk detection

  • Topology evolution — track which topologies consistently outperform others for each incident class and automatically promote them as defaults

Built With

Share this project:

Updates

Submission history