Syntrueno

Gemini agents diagnose live incidents, judge their own plans for safety, and execute real Cloud Run changes behind a cryptographic human gate. Every decision is sealed in a hash-chained audit ledger.

Comment

Inspiration

Most autonomous operations demos stop exactly where the interesting part begins. Getting a model to propose "increase the memory limit" is not the hard part. The hard part is everything that has to be true before you let it do that to a running service, and what happens when the alert it is reading was written by someone hostile. We wanted to build the version that performs the dangerous step for real, against real infrastructure, and then to be honest about every guard that makes it survivable.

What it does

An incident arrives, either raised by a person or delivered from a Cloud Monitoring alert through Pub/Sub with no human anywhere in the loop. The alert is untrusted text, so three screens run over it and their verdicts are combined. Regex rules catch the phrasings they were written for, Model Armor catches paraphrases that no regex can enumerate, and a Gemma 4 semantic screen offers a third opinion that stays advisory and never blocks. Model Armor neutralises injected instructions in place, so the evidence inside an alert survives even when the instruction buried in it does not. A Commander then mints a short-lived, scoped capability token for each dispatch. The SRE agent, running gemini-3.5-flash, diagnoses from telemetry and proposes a remediation. The Judge, running gemini-3.6-flash, scores that plan from 0 to 10 against a safety rubric. The resolved tier decides what happens next. Read-only actions proceed, a high-scoring Tier 2 action proceeds, and anything consequential stops at a D17 human gate: a SHA-256 signature bound to that exact action and parameter set, usable once, expiring after 30 minutes. Past the gate, five guards run cheapest first and fail closed. They are the project pin, the service allowlist, the verb allowlist, the destructive content screen, and the approval binding. Only once all five pass is the Cloud Run Admin API called, and afterwards the service re-reads live state until it converges rather than trusting the API's acknowledgement. Every outcome, including every refusal, is appended to a hash-chained Firestore ledger in which each entry commits to the one before it, and each entry is stamped with the trace that produced it.

How we built it

The backend is FastAPI on Python 3.14, containerised to Cloud Run, and it serves a React 19 and Vite console from the same origin as the API. Gemini 3.5-flash and 3.6-flash run through Vertex AI, using a model chain that degrades across models rather than retrying a single one. Free-tier limits are applied per model, so pooling them is what keeps the swarm answering. Model Armor screens both inbound telemetry and outbound tool calls, and Gemma 4 provides the third screening layer. Vertex AI Agent Engine, through Memory Bank, gives cross-session recall by meaning, with Firestore as the fallback that answers when the timeout expires. Firestore also holds the audit ledger and the approval store. Cloud Monitoring and Pub/Sub drive the autonomous path, and OpenTelemetry exports the swarm's reasoning to Cloud Trace, joined to the ledger by trace id. The design decision the whole system rests on is that the agent's action space is a closed enum, handed to Gemini as its response schema. There is no destructive verb inside it, so a successful prompt injection cannot produce one. The worst outcome it can reach is a wrong but safe action, which the Judge and the human gate still have to approve. No delete verb is blocked, because none exists. A blocked path can still be reached by a bug, whereas an absent one cannot.

Challenges we ran into

Vertex serves different products from different locations, and it reverses the pattern between them. Every gemini-3.x model returns 404 in us-central1 and serves from "global", while reasoningEngines return 404 at "global" and serve from us-central1. We ran into that behaviour three times before we stopped reusing a single location variable for everything. Concurrency was where the security properties actually leaked. Signing an approval and spending it are both read-modify-write operations, and FastAPI runs synchronous endpoints in a threadpool. Two executions could therefore race between the authorisation check and the consume step, both mutate Cloud Run, and leave the audit trail recording only one of them. The claim that one signature authorises one execution was simply false until we reproduced the race deterministically and closed it under a lock. The audit ledger had the same shape: two incidents arriving together both read the same chain head, both claimed the same sequence number, and the chain forked silently. The subtler problems were the ones where the service was confidently wrong. Latency was reported as the maximum of the measured value and 12.4, which meant it always reported the floor. Storage described itself as persistent while every entry was held only in memory. Tracing dropped spans on every path except the one that had been wired by hand. None of those look like failures from the outside, which is exactly what makes them worth finding.

Accomplishments that we're proud of

284 tests pass offline in about three seconds with no API key and no cloud credentials, and CI runs them on a machine that is given no secret, so the offline claim is checked rather than asserted. Every test file also runs on its own, because a suite that is green in only one order is not really telling you that the code works. The console never fabricates a figure. When the reasoning model is unreachable, the timeline reports that it ran without the model in simulation mode and shows the heuristic's own confidence, instead of printing a plausible looking number. Every latency and token count on screen was measured by the service itself. The autonomous path is genuinely real. A Cloud Monitoring alert, pushed by Pub/Sub with a verified OIDC token, was diagnosed and judged by real models and then sealed into the ledger. The Tier 3 action it produced is sitting unsigned at the gate, and the same endpoint returns 401 to a caller with no token and 401 to one carrying a forged token. Automating triage did not widen what the swarm is permitted to do.

What we learned

Making a class of failure unrepresentable is far more reliable than filtering it. Every guard we wrote as a check, we later found a way around. The ones that held were the ones where the dangerous action simply had no way to be expressed in the first place. We also learned that a system which reports its own degradation honestly earns more trust than one that always appears healthy. The work that improved this project most was not adding capability. It was finding the places where it was quietly claiming more than it had actually done.

What's next for Syntrueno

Chain integrity across containers is currently what pins the deployment to a single instance, because the ledger lock and the approval lock are both per-process. Moving both to Firestore transactions is what would allow the service to scale horizontally without forking the chain. Beyond that, we would like to widen the system from a single allowlisted canary service to a fleet with per-service policy, and to let compiled skills carry their own judge-verified safety envelope rather than inheriting the generic one.

Built With

Share this project:

Updates

Submission history