Keystone — Autonomous Grid-Wildfire Assurance [Physics-Verified Multi-Agent Fleet]

TRACK: Fortified Enterprise Fleet + Collaborative Partner on GEAP

From Idea to Impact

The Problem — A single failed component on a single tower, on a line inspected once a year by eye, can start a wildfire nobody sees coming.

The evidence:

• The 2018 Camp Fire started from one worn hook on one PG&E 115kV tower — 85 dead, the line was on a routine annual inspection cycle, not continuous monitoring.
• Utilities' own CPUC-filed wildfire mitigation plans commit to fixed, multi-year mileage targets ("underground 1,230 miles over 4 years"), not a risk-ranked annual budget.
• PG&E's own reported undergrounding cost: $3.1M/mile for ~98% ignition-risk reduction. SCE's covered conductor: $430K/mile. Nobody is asking "given this year's real budget, which specific miles buy down the most risk."
• When a line genuinely needs to come down, coordination with mutual-aid crews, fire agencies, and grid operators happens over phone calls and email today — not any interoperable protocol.

The Solution — A fleet of agents that watches the grid continuously and proves every alarm before raising it.

The core insight: detection alone isn't the hard part — an operator who gets cried-wolf three times stops looking at the fourth alert, and the fourth one is the real fire. So every finding is independently re-derived with real physics, then argued against by an adversarial agent, before anything is allowed to escalate. Only the rare, high-stakes case reaches a human — as a guided decision, not a raw alert.

What It Does

Trigger	Example	Response / Action
Imagery + telemetry anomaly	New Sentinel-2 tile + a thermal spike	Gemma triages, Gemini Vision detects, Gemini Pro fuses into one grounded hypothesis
Confirmed physics violation	Clearance drops below minimum	IEEE-738 re-derivation + adversarial critique before anything is called "confirmed"
Risk above autonomy threshold	Confirmed finding, high network centrality	Escalates to a human as a guided case, not acted on directly
Fleet risk posture changes	A pending escalation or a confirmed finding lands	Lyria generates a real alert/calm audio cue from the fleet's own live state — sonified, not decorative
Click a point on the map	"What's the clearance margin here?"	Live Gemini voice session, grounded only in that asset's real data
Engineer questions an assumption	"What if we curtail current to 400A?"	Full physics + critique pipeline re-runs live, on the spot
Field engineer attaches a defect photo	"What defect do you see here?"	Real Gemini Vision Q&A, grounded in both the photo and the case's actual physics evidence — not a generic caption
Confirmed wildfire risk	De-energization under consideration	Real Agent2Agent hand-off to 3 peer orgs — mutual aid, fire agency, grid balancing authority
Budget set by operator	Drag a real dollar slider	Exact 0/1 knapsack picks which real assets to fund, with which intervention
Natural-language question	"Which assets have the highest ignition risk?"	Ask Keystone generates and runs real SQL against live BigQuery data
Live NASA hotspot near an asset	FIRMS detection lands nearby	GKE Autopilot bursts a real Job, re-analyzes that asset immediately
Field engineer, zero signal	Any question, offline	Gemma running fully on-device via WebGPU, zero network calls
Operator approves/rejects a decision	Deliberation case resolved	Distilled into learned policy on a real 15-minute schedule — no manual trigger required

Features

✅ Feature	✅ Feature
Real ADK2 Workflow graphs (routed edges, not narrated plans)	Adversarial critique — a trust layer, not just a check
Real Agent2Agent cross-org coordination, cryptographically authenticated	Real multi-turn A2A session resumption
Self-evolving agent loop (host → judge → router → proposer)	Human approval required before any instruction goes live
Nested OpenTelemetry spans per reasoning step	Structured JSON logs with per-call token usage
CI quality gate: tests + LLM-as-judge autorater, blocking	Canary deploy + instant rollback scripts
Per-agent least-privilege identity (10 service accounts)	Verified live via impersonation, not just configured
Model Armor on ingest + Ask Keystone's free-text surface	Cloud NAT egress logging, honestly scoped (not full VPC-SC)
Fully offline on-device Gemma (Field Companion)	Real Veo + Gemini TTS briefing videos
ADK Artifact Service for case evidence photos (managed GCS)	Idempotent dispatch writes — safe against Cloud Run Job retries
Explicit Vertex AI context caching, hit/miss logged per call	429 rate-limit retried with real backoff, not treated as fatal
Real Lyria-generated fleet sonification (Grid Pulse)	Regenerated only on a real state change, not on every page load
Fully autonomous 15-minute scheduled cadence (5 Cloud Run Jobs)	Learning loop now scheduled, not manual-only
Real, managed Agent Gateway (network-layer) fronting the Controller	Distinct from the app-layer dispatch policy — governs who can reach an agent, not what an action may do

How It Was Built

Trust layer — physics_verification independently re-derives every finding (real IEEE-738 thermal model, sag, clearance), then AdversarialCritic argues against its own result before anything downstream acts on it.

Agent framework — Google ADK 2.x. The core assurance cycle and dispatch both run as real Workflow/FunctionNode/Edge graphs with routed edges on the model's own output, not hand-rolled Python if/else. A root LlmAgent Controller is deployed to a real Vertex AI Agent Engine.

Cross-org protocol — ADK's native google.adk.a2a module. Each peer is a real LlmAgent served via to_a2a(), deployed as its own private Cloud Run service, called with a real Google-signed identity token.

Evidence & memory — Vertex AI Memory Bank for per-operator/per-asset semantic recall; Google's ADK Artifact Service (GCS-backed) for unstructured case evidence — a field engineer's defect photo — queryable by a real multimodal Gemini Vision call grounded in the case's own physics evidence, not just stored and displayed back.

Grid Pulse — Lyria generates a real alert or calm audio cue from the fleet's own live risk-posture state (pending escalations, confirmed findings) — the exact same aggregate the Overview dashboard's donut chart reads — regenerated only when that state actually changes, on the same 15-minute cadence as the rest of the fleet.

Governance, two layers, not one. A real, managed Agent Gateway (networkservices.googleapis.com, Client-to-Agent mode) governs who can reach a registered agent at all — created directly against the API and verified live via an independent read after creation, referencing the project's real Agent Registry. Separately, agent_platform/gateway/policy.py is the application-layer authorization every dispatch action routes through — autonomy threshold, region isolation, Firestore-backed rate limiting. Different layers, both real, deliberately not conflated.

Backend — FastAPI on Cloud Run. Gemini Live API relayed over a WebSocket for real-time voice, grounded only in the case or asset actually on screen.

Frontend — React + Vite, no TypeScript. 15 tabs, one per capability, gated behind Firebase Authentication.

Infrastructure — BigQuery, Firestore, Cloud SQL (network digital-twin), Cloud Trace, Model Armor. Cloud Build auto-deploys on every push to main, gated by a real secret scan, the full test suite, and a blocking agent-quality eval gate.

Tech Stack

Layer	Technology
Agent framework	Google ADK 2.x — real Workflow/FunctionNode/Edge graphs, native A2A
Models	gemini-3.5-flash, gemini-3.1-pro-preview, gemma-4-26b-a4b-it-maas (cloud) + gemma-4-E4B-it (on-device), veo-3.1-lite-generate-001, gemini-live-2.5-flash-native-audio, gemini-2.5-flash-preview-tts, lyria-3-clip-preview
Backend	Python, FastAPI
Frontend	React + Vite, plain JavaScript (no TypeScript), Leaflet
Data	BigQuery, Firestore, Cloud SQL (Postgres), Cloud Storage
Interop	MCP (Google's managed BigQuery MCP server), A2A (google-adk[a2a])
Auth	Firebase Authentication
IaC	Terraform

Data Sources

Asset	Source	License
Transmission geometry (1,117 real assets)	OpenStreetMap (Overpass API)	ODbL
Imagery	Sentinel-2 L2A, Element84 Earth Search	Copernicus open data
Weather	Open-Meteo	CC-BY 4.0
Wildfire detections	NASA FIRMS (VIIRS)	Public domain
Capital-planning cost figures	PG&E / SCE / CPUC public filings	Public regulatory filings
Conductor telemetry	Synthetic, weather-driven (no public live SCADA exists)	Methodology documented in-repo

Google Cloud Services

Service	Purpose
Vertex AI	Gemini, Gemma, Veo, Lyria, Gemini Live, Gemini TTS
Google ADK 2.x	Multi-agent Workflow graphs, LlmAgent, native A2A
Agent Gateway	Real network-layer governance (networkservices.googleapis.com) fronting the Controller's Agent Engine, Client-to-Agent mode
Cloud Run	API, frontend, 5 scheduled Jobs, 3 A2A peer services
GKE Autopilot	Real-time burst re-analysis on a live fire hotspot
Cloud Build	CI/CD — secret scan, tests, eval gate, auto-deploy
BigQuery / Firestore / Cloud SQL	Findings, diagnoses, decisions / session state / network digital-twin
Cloud Storage	Evidence bucket — Veo videos, compliance filings, and operator-uploaded case photos via the ADK Artifact Service
Model Armor	Prompt-injection and PII screening on real ingest boundaries
Cloud Trace / Cloud Logging	Nested spans per reasoning step, structured JSON logs
Agent Registry API	Controller + 3 A2A peers genuinely registered

Architectural Diagram

Challenges

The Controller's dashboard showed zero traffic despite being deployed correctly. Every invocation was actually running the agent locally and using the deployed Agent Engine resource only as a session backend.

Agent Registry looked allowlist-gated for weeks. PermissionDenied on list_agents/list_services, even with the right IAM role granted directly.

Dispatch decisions replayed stale, hours-old data. A deployed session, inspected directly, showed a decision from hours earlier replaying byte-for-byte over fresh input that should have produced a different result.

The A2A SDK failed only on a clean Cloud Run container. Its own JSON-RPC routes import a package the SDK doesn't declare as a dependency — local dev's already-populated environment never surfaced it.

Every plausible model name for Veo, Lyria, Gemini Live, and Gemini TTS 404'd at least once. No naming convention predicted the real, working model ID for any of the four.

CI's evaluation gate crashed on its own first live run — before it ever graded a single real case, run-tests itself failed on a clean Cloud Build container over a missing env var import-time crash.

A push that touched nothing but documentation still failed the deploy gate. A genuine 429 RESOURCE_EXHAUSTED from heavy same-day Gemini usage during the eval gate's own grading calls, on a commit with zero code changes.

The first real evidence-photo upload failed with a session-ID error. ADK's artifact service assumes every artifact belongs to a session; a Deliberation review case isn't a conversation and has no session at all.

A live demo moment silently failed with no error shown. Asking the voice agent to open satellite imagery in a new tab visibly "succeeded" in the transcript, but nothing opened.

The feedback loop was real but not autonomous. Every consuming agent genuinely read learned policy from real data — but a direct check of the deployed Cloud Run Jobs found the distillation step itself was never scheduled, only ever run by hand.

The CLI couldn't provision the one governance product I actually wanted. gcloud's alpha/beta component groups needed write access to the SDK's own install directory, which this environment doesn't have — no agent-gateways command available at all.

A re-verification pass found a real doc gap, not just confirmations. Directly re-checking IAM state against the README's own claims turned up a tenth, undocumented service account — real and legitimately scoped, just never written down.

What We Learned

"Deployed" and "actually executing" are different claims, and only one shows up on a dashboard. Confirm the deployed resource is genuinely serving the call, not just backing a locally-run agent — a correct deployment can still hide a real gap for weeks.

An access-denied error is a claim about one code path, not necessarily about the underlying permission. The default gRPC transport failed; transport="rest" on the exact same credentials returned a real 200. Worth testing an alternate transport before concluding a wall is real.

A resumability assumption needs real production event history to validate, not code review. A deterministic session ID looked like a safe way to support crash recovery in isolation, and was silently wrong the moment a second real run happened — caught only by reading the actual event history of a live deployed session.

A dependency gap only shows up in a genuinely clean environment. Local dev had accumulated packages a fresh container never gets; the failure was invisible until the exact environment CI actually runs in was tested.

Model discovery for a fast-moving catalog has to be a live check every time, not a naming-convention guess. The real ID only ever turned up by checking each model's live Model Garden listing directly.

An evaluation gate has to actually run against a real environment to find its own bugs — including the environment it runs in, not just the logic it grades. Two separate failures (an import-time crash, then two grading-logic bugs) only surfaced once the gate genuinely executed on a clean Cloud Build container.

A CI failure isn't automatically a code regression. A transient rate limit and a genuine bug both present as "the build failed" — the retry logic has to actually tell them apart (and back off appropriately) instead of treating every non-success response the same way.

A managed SDK's default scoping assumption doesn't automatically fit a new use case. ADK assumes an artifact belongs to a session; the fix was a documented one-line convention (a "user:" filename prefix for cross-session storage), not a workaround — but only findable by reading the failure closely instead of assuming the SDK was broken.

A browser security feature can silently break something that works fine in manual testing. window.open() called from inside an async WebSocket message handler isn't in a synchronous user-gesture call stack, so Chrome blocks it with no visible error — invisible until a real, unprimed browser (not one where the developer had already clicked "allow popups") hit it live.

A feature being real doesn't mean it's autonomous — those are two separate claims to verify. Every consuming agent genuinely reading learned policy from real data proved the pipeline worked; it took a separate check of the actual deployed Cloud Run Jobs to find that nothing ever triggered the distillation step itself.

When a CLI can't reach a product, the REST API underneath it usually still can. Reading the API's own discovery document for the exact request schema, then calling it directly, worked around a local permissions wall entirely — and is a more durable fix than chasing SDK component installs.

Trust your own documentation exactly as much as you'd trust anyone else's — verify it. Re-running the same live checks that originally produced every "verified live" claim in this README, days later, is what caught the one claim that had quietly gone stale.

Accomplishments

• A live Vertex AI Agent Engine resource, independently curl-able by anyone with the right token — not a claim, a real deployed reasoning engine that has genuinely refused an action: a live mutual-aid negotiation where the grid balancing authority declined a de-energization request, with Keystone visibly blocking it on screen.
• 221 passing tests, several catching real bugs before they shipped.
• IAM segregation verified by impersonation, not just configured — a real 403 on the wrong table, a real success on the right one, re-confirmed by directly reading the live table-level IAM policy.
• A capital-allocation optimizer solved exactly, grounded in real geometry, real risk scores, and a real regulatory filing cross-check — with every cost figure independently re-verified against a live, checkable public source.
• A self-evolving loop that actually converged — 0.50 to a perfect 1.00 in two real rewrites, never applied without human approval.
• A CI pipeline that genuinely blocks a bad deploy on a real test failure, a real secret, or a real agent-quality regression — and now tells a real transient rate limit apart from an actual regression instead of failing on either.
• A field engineer's defect photo, answered on the spot — real Gemini Vision grounded in both the image and the case's own physics evidence, not a generic caption.
• A fully autonomous 15-minute cadence, now five real scheduled Cloud Run Jobs including the learning loop itself — a captured decision reaches the next diagnosis, critique, and risk ranking without anyone triggering it by hand.
• A full live re-verification pass, days after the fact — every "Managed" claim in this document re-checked directly against the running project, not re-stated from memory. One stale claim found, fixed on the spot.

What's Next

• Real per-substation load data, replacing network risk's topology proxy with a calibrated consequence model.
• A live SCADA feed once one exists for a public demo corridor.
• A broader mutual-aid network and a real integration path for an actual utility partner.
• Wiring the canary-deployment path into the default CI trigger, not a manual script.
• A blocking authorization policy on the Agent Gateway, moving it from observability-only to genuinely enforced access control.

Built With

• a2a
• bigquery
• cloud-run
• cloud-sql
• cloud-storage
• cloud-trace
• fastapi
• gemini-3.1-pro
• gemini-3.5-flash
• gemini-enterprise-agent-platform
• gemini-live-api
• gemma
• gke-autopilot
• google-adk
• javascript
• lyria
• mcp
• model-armor
• postgresql
• pub-sub
• python
• react
• veo
• vertex-ai
• vite