SriKavach AI Federation Command - Governance you can talk to

Voice+vision AI command center (Gemini Live + ADK) for governing a federated fleet of AI agents via natural conversation, with approval workflows and monitoring on Cloud Run and Firestore.

Comment

Federation Command — Governance you can talk to

Devpost submission write-up. Repo uses SriKavach AI; title uses Federation Command — Governance you can talk to.

Inspiration

Enterprises run more and more autonomous AI agents (Google, AWS, Azure, OpenAI, LangChain), but no one is watching the watchers. Dashboards assume someone is always looking; logs pile up unread. When an agent starts exfiltrating PII at 2 AM, there's no one at the screen. We were inspired by the need for governance that meets operators where they are — not another panel to stare at, but a voice-first mission control you can talk to. The name SriKavach (Sacred Shield) reflects that: one governance layer that protects every agent, in every ecosystem.

What it does

Federation Command is a voice-first AI governance command center. You speak to a Commander AI (powered by the Gemini Live API with real-time bidirectional audio and barge-in) to monitor, investigate, and enforce policy across a fleet of autonomous agents spanning multiple clouds and frameworks.

  • Talk to governance — Ask for status, pending approvals, or "what's blocking the Initech export?" and get spoken answers plus tool-driven actions (approve, reject, dispatch, purge).
  • 9-rule policy engine — First-deny-wins evaluation: jurisdiction, data classification, FOUR_EYES dual-approval, trust gate (auto-quarantine below trust 50), business hours, cross-domain, and more.
  • Cross-ecosystem — One policy engine governs agents from Google, AWS, Azure, OpenAI, and LangChain.
  • Federation — Peer "Commanders" in other domains coordinate via MCP and HMAC-SHA256 webhooks so governance works across org boundaries.
  • Interrupt cascade — One voice command ("HALT ALL AGENTS" or "Execute purge") halts all agents across ecosystems and notifies peer Commanders; auto-restore after 8 seconds.
  • Audit — SHA-256 hash-chained ledger for every decision; integrity verifiable via API.
  • Vision + image gen — Show a DPA or document to the camera; Commander analyzes it. Ask for a "compliance card" and get an AI-generated governance infographic.

How we built it

  • Voice — Gemini Live API (gemini-2.5-flash-native-audio) for real-time two-way audio; gateway WebSocket bridge in Python; 13 tool declarations so the Commander can call governance APIs mid-conversation; barge-in so operators can interrupt and get immediate acknowledgment.
  • Reasoning & tools — Gemini 2.5 Flash for tool use; Google ADK 4-agent hierarchy (Commander → Governance, Billing, Org sub-agents). Image generation via Gemini 2.5 Flash Image for compliance cards.
  • Backend — FastAPI (Python 3.11), 9-rule policy engine with first-deny-wins, review queue with FOUR_EYES and same-approver detection, SHA-256 audit chain, MCP server + client for federation.
  • Data — Cloud Firestore (native mode) for agents, policies, review queue, audit ledger; seed data for 14 agents across 5 ecosystems and 3 companies.
  • Frontend — React 18 + TypeScript + Vite + Tailwind, Three.js / React Three Fiber for the 3D agent Nexus, SSE for live events, Judge Mode (13-step guided tour), Mission Brief (8-phase cinematic), and in-app demo recorder.
  • Deploy — Cloud Run (backend + gateway), Firebase Hosting (dashboard); Docker Compose for local dev; single deploy.sh for GCP.

Challenges we ran into

  • Voice ↔ backend wiring — Getting Gemini Live tool calls to hit the real governance API (review queue, approve/reject, purge) required correct gateway routing, session affinity, and fixing a missing id on FunctionResponse so Gemini accepted tool responses.
  • Firestore and review queue — Filtered queries for "pending" review items initially hit missing composite indexes (500s); we adjusted query paths and indexes so voice prompts like "show pending approvals" work against live Firestore.
  • Frontend reliability — Approval Queue sometimes showed "Failed to fetch" due to API base URL and same-origin routing in hosted vs local; we centralized URL resolution and added fallbacks so the dashboard works consistently after refresh.
  • Policy vs UI alignment — Backend has 9 rules (PE-001–PE-009); frontend Policy Cortex had been built for 12. We aligned naming and behavior so the demo reflects the real engine.
  • Barge-in and purge UX — Making "HALT ALL AGENTS" and "Execute purge" feel immediate and visible (interrupt cascade, DEFCON overlay, 8-second auto-restore) required tight coordination between voice handling, gateway, backend purge, and SSE to the dashboard.

Accomplishments that we're proud of

  • Real voice-first governance — Not a chatbot: live bidirectional audio with barge-in, 13 tools, and real approve/reject/purge against Firestore-backed APIs.
  • One policy engine for five ecosystems — Same rules for Google, AWS, Azure, OpenAI, and LangChain; no vendor lock-in for governance.
  • FOUR_EYES for real — Two distinct approvers enforced in code; same-approver detection so one person can't double-approve.
  • Federation via MCP + webhooks — Peer Commanders negotiate cross-domain decisions; HMAC-SHA256 webhooks for external notifications.
  • Hash-chained audit — Every governance decision in a SHA-256 chain with a verification endpoint for forensics.
  • Interrupt cascade + auto-restore — One voice command halts all agents and notifies peer Commanders; automatic recovery so operators keep control without leaving the system broken.
  • Judge Mode and Mission Brief — 13-step guided tour and 8-phase cinematic built into the app, plus in-browser demo recording, so judges get a repeatable, one-click experience.
  • Shipped on GCP — Backend and gateway on Cloud Run, frontend on Firebase Hosting, live at geminiliveagentschallenge.web.app with ?mode=guided for the guided demo.

What we learned

  • Gemini Live is strong for low-latency, tool-augmented voice: warm-up and session affinity matter for demos; barge-in and tool responses need correct response shape (e.g. id on FunctionResponse).
  • Governance has to be cross-ecosystem — Teams run agents everywhere; a single, protocol-based layer (REST + MCP + webhooks) scales better than per-vendor dashboards.
  • First-deny-wins policy keeps rules understandable and auditable; combining it with trust-adaptive behavior (e.g. PE-007) gives both hard barriers and softer escalation.
  • Voice + vision + tools in one flow (e.g. "approve rq-7a3f", then show a DPA and ask "what's wrong?") makes governance feel like a single conversation instead of switching between UIs.
  • Demo UX — Judge Mode, Mission Brief, and in-app recording made it much easier to show the full story (approvals, policy, interrupt, purge, audit) without ad-hoc clicking.

What's next for Federation Command — Governance you can talk to

Q3 2026

  • Predictive trust decay (ML on agent behavior)
  • Anomaly detection (e.g. Gemini Embeddings)
  • Natural-language policy authoring ("Block PII outside EU after 6pm" → PE rules)
  • Multi-Commander federation over MCP
  • Compliance playbooks (e.g. HIPAA, GDPR, SOX)

Q4 2026

  • Self-healing policies (auto-tune from false positives)
  • Agent behavioral profiling
  • Governance-as-code (Git-backed policy versioning)
  • Real-time cost governance and budget ceilings
  • Multi-modal incident forensics (voice + vision replay)

2027

  • Governance marketplace (rule packs)
  • Agent certification and trust badges
  • Regulatory auto-compliance (e.g. EU AI Act → rules)
  • Zero-trust agent mesh with attestation
  • Cross-cloud control plane (GCP, AWS, Azure)

Federation Command — Governance you can talk to.
Built with SriKavach AI for the Gemini Live Agent Challenge.

Built With

Share this project:

Updates