CASCADE

Agent memory that knows when it has expired. Cascade learns incident runbooks from experience, reuses them instantly, and refuses any runbook the moment the policy it was built on changes.

Comment

Cascade

Agent memory that knows when it has expired.

An on-call agent that learns remediation runbooks from experience, and refuses to trust them the moment the policy underneath them changes.

What is more dangerous than an agent that does not know the answer?

An agent that knows an answer which stopped being true six weeks ago.

It is 3:14 in the morning. The pager goes off. You are on call.

You have seen this failure before. Someone on your team fixed it in April, and there is a runbook in the wiki. You find it, you follow it, you roll back the deploy.

What you did not know is that in June your team shortened the automatic rollback window from 24 hours to 4, because rolling back an old deploy took production down for nine minutes. Nobody updated the wiki. Nobody could have. The wiki does not know the policy changed.

The runbook was right when it was written. It was wrong when you ran it.

Now give that runbook to an agent, and let it fire a thousand times a day without a human reading it first. That is the failure Cascade exists to make structurally impossible.

Inspiration

  • Every agent memory system we studied was built to accumulate. None were built to forget. Vector stores, memory streams, skill libraries: all of them get bigger and more confident over time. None of them has an answer for a memory that is still perfectly retrievable and quietly no longer true.

  • In operations, confident wrongness is worse than ignorance. An agent that says "I do not know" escalates to a human. An agent that remembers a superseded procedure executes it cleanly, quickly, and in violation of a policy that exists precisely because somebody already got burned.

  • So we changed the question. Not "how does an agent remember?" but "how does an agent know when to stop trusting what it remembers?" That single reframing produced the entire design.

  • The answer turned out to be forty years old. Truth maintenance systems and belief revision theory solved dependency-directed retraction in symbolic AI decades before LLMs existed. Nobody had brought them to agent procedural memory, and nobody had put the retraction inside a distributed serializable transaction. That is the gap we filled.

What it does

Cascade is an incident response agent whose procedural memory has an expiry mechanism built into its data model. It runs a three-phase loop.

The loop

loop

  • Learn. A novel incident arrives. The agent explores step by step with Claude on Amazon Bedrock, calling tools against a governed environment. On success, a compiler distils that trajectory into a parameterised runbook together with explicit provenance edges recording every policy rule the procedure actually consulted, at the exact version it was consulted at.

  • Reuse. A similar incident arrives. CockroachDB distributed vector search finds the runbook, the provenance check confirms it is still valid, its compiled preconditions are evaluated, and the stored steps execute directly. No model is called at any point on this path. Retrieval is a vector index, freshness is a join, and the precondition check is a predicate evaluation. The same incident gets the same answer every single time.

  • Unlearn. An engineer changes a policy. One small transaction versions the rule. Every runbook derived from the old version becomes unusable the instant that transaction commits, in-flight tasks are interrupted before their next side effect, and trusted runbooks are queued for automatic re-derivation.

The part we are proudest of

Staleness is not a flag that somebody remembers to set. It is a join.

proud

  • A runbook is stale if and only if any provenance edge points at a rule version that is no longer the head version. Nothing writes the word "stale" onto a runbook when policy changes. The question is asked, and answered from data, every time a runbook is about to execute.

  • This is not a stylistic preference. It closes a real window. If invalidation is a flag, then invalidation is a background job, and a background job can be delayed, retried, or crash halfway through. The gap between "policy changed" and "flag written" is a window in which a stale runbook executes against production. In Cascade, that window does not exist, because there is no flag anywhere on the correctness path.

Watch it refuse itself

The demo moment we built everything around: shorten the rollback window, then run an incident that a stale runbook still matches perfectly.

  • Vector search finds the runbook. It is still the closest match by meaning. It still looks completely healthy.
  • The provenance join refuses it anyway, because it was compiled against rollback_window v1 while head is now v2.
  • The agent falls back to exploring, and then escalates, because that deploy is five hours old and the new window is four.
  • Two refusals, for two different reasons. It refused to reuse because the memory was stale, then refused to act because the new policy says no. A system with only the second check would have run a stale procedure and hoped the policy gate caught it.

The rest of the product

  • Autonomy gating. Irreversible actions on tier 1 services, or from runbooks that have not earned trust, park and wait for a human. Resume is by replay, which is safe because every side-effecting tool is idempotent on a deterministic {task_id}:{step_index} key.
  • Policy is data, not code. Rules carry an evaluable predicate and an enforcement mode (advisory, shadow, enforcing). You can write a rule the engine obeys without touching Python, and preview exactly which incidents it would refuse before it refuses anything.
  • Bring your own runbooks. Paste a runbook you already have. Cascade proposes which policy rules it depends on, showing the sentence each was drawn from, and nothing is written until a human confirms. Imported procedures go stale by exactly the same join as compiled ones.
  • Any agent can ask. POST /api/memory/check answers "is what I remember still valid?" with no planner, no execution and no coupling to how the caller works. Scoped, hashed, revocable keys, plus a zero-dependency MCP server so a judge with no clone can connect their editor in one command.
  • Semantic triage. Widening a window cannot invalidate a runbook that already ran inside the old one. Provably relaxing changes clear automatically, uncertain ones stay quarantined, and numeric comparison runs deterministically before any model is consulted.
  • Counterfactual replay. Before committing a policy change, re-decide every historical incident under the proposed rule and see exactly which ones would newly be automated and which newly blocked.
  • Insight engine. Proposes the smallest policy change that recovers blocked work, and only when it blocks nothing new. Computed by replay over recorded episodes, not extrapolated.
  • Time travel. AS OF SYSTEM TIME answers "what did the agent believe when it made that call", using CockroachDB MVCC directly with no event sourcing layer of our own.
  • Negative memory. Failed approaches become anti-playbooks and warn the planner, advisory only, so a stale memory of failure can never veto something policy now permits.
  • Ops Copilot. Natural language over the memory layer, answering with the SQL it ran so you can check its work.

How we built it

Architecture

architecture

  • Four processes and one database. There is deliberately no second source of truth. Staleness, history and provenance are all derived from CockroachDB rather than mirrored into anything else.

Four kinds of memory in one cluster

The schema follows the classical cognitive distinction between semantic and procedural knowledge, extended with the episodic and working memory of ACT-R-style architectures.

Memory type Table Holds
Semantic rules Versioned policy. What the organisation permits
Procedural playbooks Learned runbooks. How to act
Episodic episodes What actually happened, with outcomes and full trajectories
Working tasks In-flight state that survives a restart or an interrupt
  • playbook_deps is the connective tissue between semantic and procedural memory, and it is the table the entire product rests on.

The transaction that makes forgetting cheap

transaction

  • We rejected the obvious design. One large transaction that versions the rule and mass updates every dependent runbook creates an unbounded write set, and under serializable isolation it contends directly with the table the retrieval path is reading. That produces retry storms at exactly the worst moment.

  • Deriving staleness instead of writing it makes the cascade O(1) in the size of the blast radius. Sixteen times the dependent set, and the cascade is not slower. The difference between those two rows is network noise, because neither transaction touched a single runbook row.

  • Freshness for one runbook stayed at roughly 275 ms across both scales, since it reads that runbook's own provenance edges and nothing else.

CockroachDB tools we used, and what the agent did with them

  • Distributed Vector Indexing. The core retrieval path. Runbooks are embedded with Titan v2 at 1024 dimensions into a VECTOR(1024) column with a C-SPANN index, and every query uses the L2 operator to match the index metric. This is the hot path, not a side feature.
  • Managed MCP Server. How we explored the schema and verified query plans during development, and the reason we caught the vector index defect described below.
  • ccloud CLI. Provisions the cluster, in infra/01_ccloud_provision.sh.
  • Agent Skills Repo. Produced a schema and performance review that turned into twelve findings, every one of them a live defect rather than a hypothetical. All of them were fixed and are written up in docs/skills-review.md.

AWS services we used, and how

  • Amazon Bedrock. Claude Sonnet 4.6 plans the explore loop and compiles trajectories into runbooks, Claude Haiku 4.5 handles short structured calls, and Titan Text Embeddings v2 produces the 1024-dimensional unit vectors that the index geometry depends on.
  • AWS Lambda. The background worker: compile, rule_changed, relearn, recheck_suspect, postmortem and insight_scan.
  • Amazon SQS. Carries outbox events from the API to the worker.
  • Amazon EventBridge. A 60-second sweeper that makes the transactional outbox correct rather than hopeful.
  • Amazon ECS Fargate. Runs the API behind an ALB.
  • Amazon S3. Stores full episode trajectories.
  • Amazon CloudFront and AWS Amplify. HTTPS, and the frontend built against it.
  • AWS Secrets Manager. Database credentials and tokens, never in the client bundle.

Engineering decisions worth defending

  • Retrieval is two statements, deliberately. Phase one is a pure approximate nearest neighbour query carrying no predicate at all. Phase two re-reads the winners by primary key and applies the metadata filter. Combining a vector ORDER BY with any scalar filter makes the planner abandon the index, and we have the EXPLAIN proof of both plans committed in the repo.

  • Provenance is grounded, never asserted. A model asked to summarise a run will happily cite a plausible-sounding rule it never saw. Every citation is cross-checked against what the run actually observed: the policy snapshot it read, and the rule versions the eligibility check reported using. A citation corroborated by neither is dropped, because an invented edge would point at a rule the runbook does not really depend on, and the runbook would then look fresh forever.

  • Policy is enforced by the tools, not by the model. apply_remediation re-reads head rules and refuses on its own. The agent cannot talk its way past a policy, because the policy check is not part of the conversation.

  • Interrupts have three layers, and only one of them is authoritative. An in-process bus delivers in microseconds and an SNS broadcast reaches peer instances in about a second, but the durable tasks.interrupt_flag, checked immediately before every side-effecting call, is the guarantee. Correctness never depends on the fast path.

  • Survivability is switched on and readable from the app. The database runs SURVIVE ZONE FAILURE with three voting replicas placed in separate AWS availability zones. Losing an availability zone costs a leaseholder re-election and nothing else. The Architecture view reads SHOW SURVIVAL GOAL and SHOW ZONE CONFIGURATION live from the cluster rather than asking you to believe a diagram.

Research foundations

Cascade's core mechanism is not a new idea. It is a very old one, applied where it had not been applied before. Each of these maps onto a specific piece of the implementation.

  • Doyle (1979), "A Truth Maintenance System." Dependency directed backtracking: record why you believe something, and when a justification is retracted, everything resting on it loses support automatically. Our playbook_deps table is a justification network, and the freshness join is dependency-directed backtracking evaluated lazily at the point of use.

  • Alchourrón, Gärdenfors and Makinson (1985), "On the Logic of Theory Change." The AGM theory of belief revision formalises contraction: retracting a belief and everything depending on it while disturbing the rest of the knowledge base as little as possible. Our four-write cascade is a minimal change contraction over a procedural knowledge base.

  • Tulving (1972), "Episodic and Semantic Memory." The distinction our schema is literally built on, extended with the procedural and working memory of ACT-R-style cognitive architectures (Anderson, 1996).

  • Wang et al. (2023), "Voyager: An Open-Ended Embodied Agent with Large Language Models." Demonstrated that an LLM agent can build a reusable skill library from experience. Voyager's library grows monotonically. Cascade adds the operation it is missing: principled removal when the world invalidates a skill.

  • Shinn et al. (2023), "Reflexion" and Park et al. (2023), "Generative Agents." Verbal reinforcement, and memory stream retrieval with recency and importance weighting. Both accumulate. Neither has a mechanism for a memory becoming wrong because something external changed.

  • Packer et al. (2023), "MemGPT: Towards LLMs as Operating Systems." A memory hierarchy with paging between context and external storage. MemGPT manages memory capacity. Cascade manages memory validity, which is an orthogonal and unaddressed axis.

  • Yao et al. (2022), "ReAct: Synergising Reasoning and Acting in Language Models." The interleaved reason and act loop our explore mode follows.

  • Chen et al. (2021), "SPANN: Highly-efficient Billion-scale Approximate Nearest Neighbor Search" (NeurIPS). The memory and disk hybrid ANN design behind CockroachDB's C-SPANN vector index, which is what makes phase one retrieval fast enough to sit on the hot path of every incident.

  • Taft et al. (2020), "CockroachDB: The Resilient Geo-Distributed SQL Database" (SIGMOD). Serializable isolation across a distributed cluster is precisely what lets the cascade be four rows and still be correct under concurrent executors.

  • The gap we fill, stated plainly. The agent memory literature is about acquisition and retrieval. The belief revision literature is about retraction, and predates LLMs by decades. Cascade connects them, and puts the result on a production database where the retraction is a transaction rather than a theory.

Challenges we ran into

  • A real model overfits preconditions in a way a stub never will. Our first compile on a live LLM produced the precondition "the incident is of severity P1". The demo incident is P1 and the reuse incident is P2, so the runbook matched on retrieval and then refused itself. Reuse silently died and the headline demo step went cold. The model had described the incident it saw rather than when the procedure applies. We rewrote the compiler prompt to forbid encoding incidental properties, and then went further: preconditions are now compiled into a checkable predicate, validated structurally and behaviourally at compile time, so reuse never depends on a model re-reading English at run time.

  • A precondition checker that could not evaluate its own precondition. It was asked to verify "the deploy occurred within the rollback window" while being handed only incident data. The window lives in the rules table. Unable to verify, it answered false, every reuse fell back to explore, and the measured speedup collapsed below 1x. The fix was to pass head rules into the check and to recognise that this is a routing decision, not a safety gate. Policy is enforced independently and downstream.

  • A wrong parameter name that would have passed forever. The model's first compiled predicate cited auto_remediate_tier.max_tier when the real parameter is min_tier. It resolved to nothing, compared against nothing, evaluated to unknown, and was treated as satisfied. It looked like a working gate and checked absolutely nothing. Compiled predicates are now validated against the real parameter set, and must additionally hold for the incident they were learned from, which is the closest thing to a unit test a compiler can run on its own output.

  • One innocuous predicate cost us the vector index. The phase one query carried WHERE embedding IS NOT NULL. That alone made the optimiser abandon pb_embed_idx and full scan with a top-k sort. The answers stayed correct, which is exactly why it went unnoticed. We committed both query plans, including the failing one, because the failure is more instructive than the success.

  • Guided mode ignored its own eligibility check. It called check_remediation_eligibility, recorded the answer, and then ran apply_remediation regardless. Explore mode was safe because the planner reads results. Guided mode replayed steps mechanically. A tier 2 incident outside the rollback window would have been remediated in direct violation of policy. Found while building autonomy gating, fixed, and regression tested.

  • The admin token was being published in the page source. Anything prefixed NEXT_PUBLIC_ is inlined into the client bundle at build time, and our deploy script was reading the token out of Secrets Manager in order to put it there. It took a managed secret and made it public while looking secure. Fixed with a server-side proxy carrying an explicit path allowlist.

  • A re-learn that produced weaker provenance than what it replaced. A v2 compiled from an escalation cited fewer rules than v1 did, so it would have survived the very rule change that quarantined its predecessor. The runbook would have looked healthy while resting on policy nobody had checked. The compiler now refuses any replacement that drops a rule which has actually moved.

  • Nine defects in deployment scripts that had never been executed, including an S3 call that could not create a bucket in the project's own pinned region, IAM permissions granted to the wrong ECS role, and a CloudFront configuration that buffered the event stream so the dashboard received nothing at all.

Accomplishments we are proud of

  • The unlearn guarantee is real and enforced, not asserted. A stale runbook cannot execute, even in the seconds before any cache catches up, because correctness never depends on a cache.

  • 11.45x measured on the deployed stack, not estimated. Cold 13,158 ms to guided 1,149 ms on Amazon Bedrock, with 7,469 planner tokens avoided per reuse, down to exactly zero. The token figure is structural rather than a measurement: a reuse calls the planner zero times.

  • The cascade is provably independent of blast radius. 3,000 dependent runbooks: four writes. 50,000 dependent runbooks: four writes, and not slower. The integration suite asserts the write count, which is the actual property, rather than a wall clock, which is the network.

  • 109 assertions passing, zero failures, against a live CockroachDB Cloud cluster on real Bedrock models. The suite refuses to run in stub mode, so a green result can never be a canned one. It talks to the engine directly rather than over HTTP, because the interrupt case needs a task already carrying its flag before execution starts.

  • The vector index proof is committed, including the full scan plan that one stray predicate produced, and re-proven on the multi-node Cloud cluster rather than only on a laptop.

  • Zone survival is switched on and verifiable from inside the running app. Three voting replicas across separate availability zones, read live from the cluster, not claimed in a document.

  • The memory layer is usable without adopting our agent at all. Import your own runbooks, write your own policy rules, and let your own agent call /api/memory/check over HTTP or MCP. Those three are the product. Our agent is just one consumer of it.

What we learned

  • Stubs hide the failures that matter. Three of our most serious bugs were only reachable with a real model writing real output. A deterministic fallback planner builds preconditions from a fixed template and will never overfit, so it will never show you the class of bug that kills reuse in production.

  • "Fail closed" is not automatically safe. Our precondition checker failed closed and destroyed the entire value of the system while protecting nothing, because the real safety gate was two layers below it. Knowing which layer is load-bearing matters far more than defensive instinct.

  • Deriving beats writing. Almost every hard problem got easier the moment we stopped storing a fact and started computing it. Staleness as a join instead of a column removed a write contention bottleneck, an entire class of race conditions, and the possibility of a missed update, all at once.

  • Provenance you cannot trust is worse than no provenance. A fabricated citation does not fail loudly. It quietly makes a procedure look governed while never being able to go stale, which is the exact failure the system exists to prevent, wearing the system's own uniform.

  • The interesting failures are the silent ones. A stray predicate that drops an index still returns correct answers. An overfitted precondition still produces a valid run. A weaker re-learn still produces a working runbook. Every one of those was caught by asserting a mechanism rather than an outcome, which is why the suite checks the query plan, the write count, and the provenance set.

What is next for Cascade

  • ** Cross-domain provenance.** Nothing about the mechanism is specific to incident response. Any agent whose procedures depend on versioned external facts has this problem: compliance workflows, pricing rules, clinical protocols, tax logic. The domain is a configuration surface, and domain_facts is already where it plugs in.

  • Real integrations behind the same policy gate. Slack, Discord and webhooks already receive real notifications with replay suppression proven against the ledger. The next step is remediation targets, with the idempotency ledger doing the work that an Idempotency-Key header cannot be trusted to do.

  • Authentication. Cascade has authorisation with three ordered roles enforced on every write endpoint. It does not have authentication. Cognito or OIDC in front of CloudFront with the principal resolved from a verified JWT is the next step, and the Principal seam was designed around exactly that swap.

  • Multi-region survival. Zone survival is in force today. The per-table localities are already written and reasoned about: GLOBAL for policy and procedures because they are read constantly and written rarely, REGIONAL BY ROW for operational data. Turning it on is one statement once the cluster spans three regions.

  • Multi-tenancy, done properly or not at all. It needs an organisation column on every table and scoping in every query. Half-done multi-tenancy is a data leak vector rather than a partial feature, so we deliberately did not start it.

Why this should place first

Most submissions in an agentic memory hackathon will demonstrate an agent that remembers. Cascade demonstrates an agent that knows when its memory has expired, and refuses to act on it.

That difference is not cosmetic. Remembering is the easy half, and it is already a solved commodity: embed, index, retrieve. Knowing when a memory has stopped being true is the half that decides whether an agent is safe to leave running unsupervised in production, and it is the half nobody builds.

Against the judging criteria, concretely:

  • Agentic Memory Design. Four distinct memory types in one cluster, connected by a provenance graph, with a lifecycle that includes principled forgetting. CockroachDB is not a vector store bolted onto the side of this project. The serializable transaction is what makes the invalidation correct, the distributed vector index is what makes retrieval viable on the hot path of every incident, and MVCC is what answers "what did the agent believe when it made that call" with no event sourcing layer of our own. Remove CockroachDB and the idea does not survive the substitution.

  • Technical Implementation. Two-phase retrieval with a committed query plan proof on the Cloud cluster. An O(1) cascade validated at 50,000 dependents. A transactional outbox with an idempotent claim and a sweeper that makes it correct rather than hopeful. Exactly once side effects on a deterministic key. Three-layer interrupts where only the durable layer is authoritative. 109 assertions passing against live models, with the suite refusing to run against stubs.

  • Real World Impact. Every on-call engineer has met a stale runbook, and a stale runbook executing against production infrastructure is an outage multiplier rather than a nuisance. We cut repeat incident handling by 11.45x and remove the planner from the reuse path entirely, without trading away the safety that makes automation acceptable in the first place. And it works on your material, not only ours: import your existing runbooks, write your own rules, connect your own agent.

  • Production Readiness. Scoped SQL roles with append-only audit enforced by grant rather than convention. Circuit breakers, budget ceilings, idempotency, a credential that never reaches the browser, an audit trail that survives a demo reset, row-level TTL for retention, OpenTelemetry tracing, and zone survival switched on and readable live from the app. Twelve documented findings from the CockroachDB Agent Skills review, all fixed. Sixteen documented deviations from our own specification, each with rationale and impact.

  • Creativity and Originality. Truth maintenance and belief revision are decades-old ideas from symbolic AI, and distributed serializable transactions are a decades-old idea from databases. Connecting them so that an LLM's learned procedure can be retracted correctly, atomically, and in constant time is, as far as we can find, new. That connection is the contribution, and everything else in the repository exists to make it checkable.

And here is what Cascade does not claim. It does not claim zero hallucination. It claims something narrower, sharper and testable: a runbook whose provenance is stale cannot execute. That is a guarantee you can falsify in about ninety seconds, and we wrote 109 assertions that try.

Repository: open source, MIT licensed, with a ten-minute local quick start that needs no cloud account and no API keys.

Built With

  • amazon-bedrock
  • amazon-cloudfront
  • amazon-ecr
  • amazon-ecs-fargate
  • amazon-eventbridge
  • amazon-sns
  • amazon-sqs
  • amazon-titan-text-embeddings-v2
  • amazon-web-services
  • aws-amplify
  • aws-iam
  • aws-lambda
  • aws-secrets-manager
  • cockroachdb
  • cockroachdb-agent-skills
  • cockroachdb-cloud
  • cockroachdb-distributed-vector-indexing
  • cockroachdb-managed-mcp-server
  • docker
  • fastapi
  • next.js
  • node.js
  • typescript
  • uvicorn
Share this project:

Updates