Aftershock

Inspiration

Runbooks describe the system we intended to build. On-call engineers need the history of the system that actually failed: the signal, the observations, the actions taken, and what finally restored service. That history is usually scattered across chat logs and ticket timelines.

What it does

Aftershock opens a live incident, retrieves semantically similar resolved incidents, and turns their evidence into a cautious action plan. Every action shows confidence, risk, and the precedent behind it. New observations update the hypothesis. When the incident is resolved, the complete timeline becomes durable memory for the next on-call engineer.

How we built it

• CockroachDB stores live incident state, timelines, and resolved memories.
• Distributed Vector Indexing performs service-scoped cosine recall over deterministic 64-dimensional embeddings.
• Cockroach Labs' official cockroachdb-sql Agent Skill guided and audited the schema and query design; the exact upstream blob is pinned in the repository.
• FastAPI implements the agent workflow and API.
• AWS Lambda and HTTP API provide the public compute layer through AWS SAM.
• A zero-cost in-memory mode keeps the demo and test suite reproducible without cloud credentials.

Challenges we ran into

The most important design constraint was avoiding false confidence. A similar incident is evidence, not proof. Aftershock filters weak matches, exposes the similarity score, distinguishes low- and medium-risk actions, and falls back to reversible diagnostic steps when no precedent is strong enough.

The second challenge was serverless durability. The first prototype kept live incidents in process memory. The production path now persists active state and every observation in CockroachDB, so a fresh Lambda instance can continue the same incident.

Accomplishments that we're proud of

• One workflow connects live triage and durable organizational memory.
• Vector recall and operational state share one distributed SQL system.
• The app remains useful with deterministic local tests and no paid model API.
• The repository records exactly which official Agent Skill revision shaped the SQL, making the second CockroachDB tool independently auditable.

What we learned

Agent memory is most valuable when its provenance is visible. A concise answer with a linked precedent is safer and more actionable than a confident answer whose history is hidden.

What's next for Aftershock

Add human approval gates for high-risk actions, import incident timelines from authorized observability systems, and use multi-region locality so an outage in one region cannot erase the memory needed to repair it.

Built With

• aws-api-gateway
• aws-lambda
• cockroachdb
• cockroachdb-agent-skills
• cockroachdb-vector-search
• fastapi
• python
• remotion