OpsVoice

OpsVoice

Inspiration

On-call incident response is one of the worst moments to rely on a text-only assistant. When a P1 fires at 3 AM, you're juggling terminals, dashboards, Slack, and PagerDuty -- the last thing you want is to type detailed queries into a
chatbot. Engineers need their hands free. They need an assistant that can hear them, see their screens, and take action -- not another text box.

What it does

OpsVoice is a real-time, voice-first SRE command center that lets on-call engineers diagnose, triage, and respond to
production incidents using natural voice conversation, camera/screenshot vision, and text -- all without touching a
keyboard.

The agent has a distinct SRE persona: it speaks like a senior Site Reliability Engineer -- concise, direct, and
action-oriented. It always cites which tool provided the data.

Core capabilities:

Voice-first interaction: Bidirectional audio streaming with Gemini 2.5 Flash native audio -- sub-second latency,
no text-to-speech conversion
Natural interruption (barge-in): Speak while the agent is talking -- it stops, listens, and pivots immediately
Proactive alerting: A background health monitor detects service degradation and broadcasts voice alerts via
WebSocket -- before you even ask
Incident lifecycle management: Create, track, update, and resolve incidents entirely via voice, persisted in
Firestore
Vision and screenshot analysis: Camera feed at 1fps + drag-and-drop screenshots for multimodal troubleshooting
Runbook retrieval: Context-aware remediation steps with tool chaining
Google Search grounding: Real-time search for unfamiliar errors and CVEs -- preventing hallucination

How I built it

Gemini 2.5 Flash Native Audio via the Live API for bidirectional audio streaming
Google ADK with Runner.run_live() and LiveRequestQueue for real-time agent orchestration
FastAPI async WebSocket backend on Google Cloud Run (3600s timeout, 0-2 auto-scaling)
Google Cloud Firestore for persistent incident storage
Secret Manager for secure API key management
Terraform + deploy-gcp.ps1 for fully automated infrastructure provisioning
Vanilla JS browser client with Web Audio API, AudioWorklet, and adaptive Voice Activity Detection

Data sources

Service health simulation: Built-in engine generating realistic metrics for 6 microservices with evolving degradation scenarios. Same tool interface works with real monitoring APIs (Datadog, Prometheus, PagerDuty).
Incident records: Persisted in Google Cloud Firestore, managed via voice commands
Runbooks: 5 predefined runbooks covering common SRE scenarios
Google Search: Real-time grounding for unfamiliar errors and CVEs
User input: Voice (16kHz PCM), text, camera frames (JPEG), drag-and-drop images -- streamed in real-time, never
persisted

Challenges I ran into

Streaming audio reliably through browser AudioWorklet, WebSocket, and Cloud Run while maintaining sub-second latency
Implementing adaptive Voice Activity Detection that works across different microphones and environments
Designing the proactive alert system so the agent pushes warnings without interrupting ongoing conversation
Coordinating barge-in handling across the WebSocket pipeline -- flushing audio playback and signaling the backend
simultaneously

What I learned

Gemini's native audio model enables remarkably natural voice interactions -- it genuinely feels like talking to a
colleague
Google ADK with Runner.run_live() dramatically reduces the complexity of building streaming agents with tool calling
Voice Activity Detection is crucial for voice UX -- adaptive thresholds beat fixed ones every time
Proactive agent behavior fundamentally changes how useful the agent feels -- less like a tool, more like a teammate
Streaming UX is unforgiving -- every millisecond of latency matters

What's next

Replace the built-in health simulation with real monitoring APIs (Datadog, PagerDuty, Prometheus)
Multi-region Cloud Run deployment with Firestore global distribution
Multi-user incident channels so multiple on-call engineers can share the same voice session
Audio playback buffering optimization for higher-latency networks

Automated deployment

Fully automated with Infrastructure-as-Code:

Terraform provisions all GCP resources (Cloud Run, Firestore, Secret Manager, Artifact Registry, IAM)
deploy-gcp.ps1 performs one-click build, deploy, and health check
Templates (.env.example, terraform.tfvars.example) included for easy reproduction