Cascade

Most agent guardrails ask if an agent is allowed to act. Cascade asks what happens if we let it. It rehearses the work, diffs the result, and decides whether it becomes real.

Comment

An agent refunded someone twice.

It was allowed to. It held payments.refund, granted by the team that owns refunds. The customer really had withdrawn. The amount was right. Every policy check came back green.

The problem was that we'd already refunded her five months earlier. Nothing in the request said so. There was nothing odd about the request at all. What made it wrong wasn't in the request, it was sitting in the database.

I went looking for something that would have caught it and couldn't find one. Every agent guardrail I read about answers the same question: is this agent allowed to do this? Capability checks, allowlists, policy engines, gateways. They all look at the request. That's worth having, and it catches a compromised agent or a badly configured one.

But it doesn't catch the agent that was allowed to do exactly what it did and got it wrong anyway. That's most of what actually goes wrong in production.

What it does

Cascade asks a different question. Not "can it?" but "what happens if we let it?"

It takes a copy of your world. Real records, not mocks. The agent does its work in the copy, calling real tools against real state, unable to reach anything outside it. Then Cascade compares before and after, and judges what actually changed. Not what the agent said it was going to do. What it did.

Once you're looking at the result, "this customer gets a second refund" is just a fact sitting there. You didn't have to think of it in advance.

You don't change the agent to use this. You point its base URL at Cascade and its calls get judged on the way through. Systems get declared rather than integrated: a URL, the name of a secret, and what each tool does.

And when you say a tool is reversible, Cascade doesn't take your word for it. It runs the tool, runs the inverse you declared, and checks the record came back. If it can't prove that, it treats the tool as irreversible and makes a person look. An unproven claim costs you an approval. It never costs you something you can't undo.

Everything gets sorted by whether you can take it back, not by how risky it looks. I tried risk scoring first. It turns into constant interruptions, wrong in both directions, and people stop reading them.

Reversible work goes through on its own, with the undo recorded before it runs. Compensable work, like a refund you can offset later, waits. Irreversible work always waits, however harmless it looks.

One goal, three different answers:

Goal: Refund the enrollment fee for everyone who is no longer proceeding

BLOCK Ngozi Okonkwo A fee may be refunded at most once. A second outstanding refund is a duplicate payment. GATE Seyi Bakare Over the approval threshold, needs a named approver GATE Blessing Udo Over the approval threshold, needs a named approver

That sorting is also why it scales. More agents means more throughput, most throughput is reversible, and reversible work never reaches a person. I measured it: take a fixed amount of work and spread it across nine agents or a thousand, and the number of decisions landing on a human doesn't budge. How big that number is depends on what your agents do. That it doesn't grow with the fleet is the part I actually measured.

How I built it

Cloud Run runs the control plane. It scales sideways and costs nothing while a workflow sits waiting.

Firestore holds the ledger, the approvals, the world state. This one mattered more than I expected. Cloud Run gives you a lot of instances, and a SQLite file inside one of them is durable right up until the revision gets replaced. Firestore gives every instance the same view, partitioned by organization. The ledger gets written before anything runs, so a crash leaves you a decision with no effect, never an effect nobody decided.

Vertex AI serves Gemini for breaking goals down and adjudicating, scoped by IAM instead of an API key sitting in an environment variable.

Google's ADK runs the workflows, and it earns its place on gates. An approval might take two weeks. No held process survives that, but a durable ADK session picks up on any instance with its context intact.

Model Armor screens goals and retrieved records before a model reads either. Agent Registry is where the fleet lives, so nothing about it is hardcoded and you find an agent by what it can do rather than by name. Memory Bank carries refusals between sessions. Cloud Trace gets one span per decision. Secret Manager holds every connected system's credential, injected per call, never handed back by a route.

The rule I refused to bend: no model in the decision path. The model proposes, deterministic Python decides. Same facts and same rules give the same answer today and in an audit eight months from now, and every action carries a hash of the rules that judged it.

Challenges I ran into

One mistake, five places, and I only found it by pointing this at a real API.

Cascade uses an expression like jobmatch:{id} to spot two plans writing the same record. It's a conflict key. It is not a URL. In five different places the code went and fetched it as if it were one. Proving did it. The rehearsal did it. The reader even invented a URL from it.

All five were broken. All the tests passed. The fakes were keyed the same wrong way, so they agreed with the bug.

I didn't find any of it until I connected my own production API. Until then, reversibility couldn't be proven for any real system, and unproven means everything waits for a human, so it would have looked like strict policy rather than a broken lookup.

A cost endpoint that could never have worked, and a fix that didn't either. /api/metrics/cost returned zero because getattr(run, "tokens_used", 0) was the only mention of that field anywhere and nothing ever set it. So I instrumented the ADK event stream. Tidy, obviously correct, still zero, because most of the spend goes through direct client calls that never emit an ADK event. I only found that by running a real goal against Vertex instead of trusting a green suite.

Role checks that turned everyone away. My JWT had no role claim, so user.get("role") was None for every single caller and every gate refused everybody, me included. It failed closed. That's exactly what a working security control looks like from the outside, so nothing broke loudly enough to notice.

A test that matched attributes where the product matches meaning. I added records to the demo world and wrote a guard saying the refund goal should match three candidates. Green. Then a live run came back with four. Two of my records were stage: rejected but still holding an unrefunded fee, which Gemini correctly read as "no longer proceeding" and my attribute check did not. If you put a model in your system, that's the class of bug I'd warn you about hardest.

A deploy command in my own README that would have taken production down. It named a service that doesn't exist and passed --set-env-vars, which replaces the whole environment instead of adding to it. Running it would have dropped CASCADE_LEDGER=firestore and quietly moved the ledger into a file inside the container. The service would have started fine. Health checks would have passed.

What I learned

Not one of those was visible to a passing test suite, and they all failed differently. A check that couldn't see the files most likely to break it. A control that failed closed, so its failure looked like it working. A default that made missing data look like real data. A test matching attributes where the product matches meaning. Docs nobody had ever run. Test doubles built on the same misunderstanding as the code.

Same thread through all of them. My tests agreed with my mental model, and my mental model was wrong. Tests can't catch that. The same person wrote both.

Two habits found every one, and neither is clever. Read the diff before you commit, not the list of files. And run the real thing against the real service before you believe it works, especially when the suite is green and the fix looked tidy.

What's next

Cryptographic attestation for agents, so a decision can rest on identity rather than on a declaration. Authority tiers on approvals, which right now are only protected by not letting you approve your own work. Payments, because usage is metered properly and nothing charges for it yet. And a real fleet run, since the scaling number was measured mechanically rather than with a thousand agents doing actual work.

All of those are printed as not met by the compliance check that ships with the project, right next to the eighteen things it verifies against the live deployment. A report that only lists what you passed is a brochure. Naming your own gaps is what makes the rest of it worth believing.

Agents aren't held back by how clever they are. They're held back by consequence.

Permission is necessary. Consequence is decisive.

Built With

Share this project:

Updates