Tele Governed Agents (TGA).

Inspiration

Call centres process thousands of sensitive conversations. A single unconstrained LLM endpoint hides: which agent ran, which model/memory/policy revision was used, what happens when reviewers disagree, whether a failed job can resume, and whether the model can directly mutate state. Tele-Governed makes those boundaries explicit and verifiable — not with a dashboard, but with durable jobs, policy gates, and hash-bound records.

Original seam: on Vertex AI Agent Engine + Memory Bank, a byte-identical quarantine request returned invoice-classifier → billing-reconciler after one memory correction (evidence/h4b-*.sse). The trace kept text, not memories/{id}/revisions/{rev}. We record the retrieval boundary so the move is explainable.

What it does

Live: https://tele-governed-597227190850.us-central1.run.app — console + API one origin. Dedicated Tele view: /tele

Approved GCS audio (TELE_APPROVED_AUDIO_URIS allowlist)
  → POST /api/tele/batches → Firestore tele_jobs (queued first)
  → Pub/Sub telephony-batches → private Cloud Run worker (/api/tele/pubsub, IAM-protected, TELE_ENABLE_PUBSUB_PUSH)
  → tele-ingestor (ASR)
  → tele-qa-analyst (Gemini 3.5 Flash, 4 claimant roles: compliance / customer-harm / evidence-skeptic / operations)
  → commit → reveal → code verifies spans, detects incompatible classification/remedy, deterministic adjudication
  → awaiting_human for critical/quarantine/delete (exact remediation envelope bound to SHA-256)
  → tele-report-remediator → Firestore report + decision ledger + correction ledger

• Resumable: BatchWorker persists each stage; retryable on transient ASR/Gemini failure resumes after last durable stage, not re-transcribing. Proven: evidence/video-rehearsal.json job d348f36b… attempts 6: queued → published → transcribing → failed (bucket allowlist) → transcribing → analyzing → disputing → governing → awaiting_human
• Dispute: typed EvidenceSpan (wav,start_ms,end_ms,text,sha256) ops/disputes.py, commit = sha256(reveal), verify_ruling checks hashes, pure adjudicate() with ROLE_PRIORITY evidence-skeptic > customer-harm > compliance > operations and INCOMPATIBLE_REMEDIES.
• ASR routing: deterministic inspect_wav + fingerprint (customer, codec, noise band, accent, language_mix) + choose measured WER + 0.15*calibration with min_samples floor and shadow scoring on correction ops/asr_routing.py.
• Remediation: make_envelope binds job, target revision/state, action, payload, evidence hashes, policy revision, retrieval hash, ruling hash, expiry 15m; APPROVAL_EXPIRED if target changed, edit creates SUPERSEDED new envelope ops/remediation.py.
• Verification: GET /api/tele/batches/{job}/dispute/verify + /api/decisions/{hash}/verify capability BOUND_UNCORROBORATED / BOUND / UNBOUND / NOT_CERTIFIED ops/verifier.py — BOUND only with exported Cloud Trace bundle (scripts/export_bundle.py).

How we built it

• Agent Engine: Gemini 3.5 Flash via google-cloud-aiplatform[agent_engines,adk], deploy/agent_engine.py (now with --gateway Agent Identity + AGENT_IDENTITY).
• Memory Bank: VertexMemoryBank for retrieval manifest ops/retrieval.py — fail-closed unless GAO_ENGINE_ID set, TELE_ALLOW_DEMO=1 labeled mode=demo.
• API/Worker: FastAPI + uvicorn ops/service.py one image, two deployments (public vs TELE_ENABLE_PUBSUB_PUSH worker with Cloud Run IAM as per-route guard).
• State: Firestore (fleet, jobs, ledger, corrections, routing measurements), Pub/Sub publisher ops/jobs.py, CloudRunASRTranscriber ops/jobs.py:152.
• Hardening: Model Armor TELE_MODEL_ARMOR_TEMPLATE, Agent Registry/Gateway/Identity + IAP egress grants (deny canary 403), OpenTelemetry ops/telemetry.py — all reported via GET /api/tele/evidence only when TELE_*_EVIDENCE file present (content-bound, not config text).
• Tests: PYTHONPATH=. pytest -q 237 passed + frozen tele-ci / router-ci 4-arm benchmarks.

Challenges we ran into

• Memory scope must be {app_name,user_id} exactly or it’s invisible to the agent — debugged via scripts/memories.sh:3.
• Global GEMINI_LOCATION=global 429 while europe-west2 200 — made model location a deploy var.
• Pub/Sub push IAM applies to whole service, not one route — split public vs private worker.
• verify honesty: manifest built by executor’s call, not observed agent retrieval → BOUND_UNCORROBORATED until trace exported.

Accomplishments that we’re proud of

• Real async pipeline that survives failure and proves it (not a demo queue).
• Code, not prose, settles disputes and binds approvals.
• No claim without artifact: GET /api/tele/evidence refuses unverified controls; wer-evidence.json gate keeps accuracy claim false until benchmark passes.

What we learned

Audit boundary is revision + population, not just text. Falsifiability matters: ablation identities stripped → UNBOUND, population stripped → UNBOUND, manifest removed → NOT_CERTIFIED proves the binding is load-bearing.

What's next

• Replace seeded readiness evidence with deployment-sourced checks (still per-exact-revision, never inherited).
• Promote router policy via frozen benchmark gate only.
• Expand approved audio set beyond demo smoke object with owner permission.

Built With

• agent-gateway
• agent-identity
• agent-registry
• fastapi
• gemini-3.5-flash
• google-adk
• google-cloud
• google-cloud-firestore
• google-cloud-pubsub
• google-cloud-run
• google-cloud-trace
• iap
• model-armor
• opentelemetry
• python-3.12
• vertex-ai-agent-engine
• vertex-ai-memory-bank