SupportSight Live

Multimodal incident agent for SRE/DevOps: voice, screenshots & logs in one flow. Gemini-powered triage with human-in-the-loop.

Comment

Inspiration

When a production incident hits, engineers face scattered logs, error screenshots, and Slack noise—with little time to think. We wanted an agent that could see (screenshots), hear (voice), and read (logs) in one flow, so SREs and DevOps can triage faster with AI that reasons multimodally and never acts without human approval.

What it does

SupportSight Live is a multimodal incident command center. The user describes the incident (by voice or text), optionally attaches a screenshot and pastes logs. The agent analyzes everything with Gemini, returns ranked hypotheses, a root-cause summary, and suggested actions (e.g. restart service, check endpoint). Every action requires explicit approval (human-in-the-loop). Sessions are stored (Redis or in-memory fallback) and can be audited.

How we built it

  • Web: Next.js 15 with voice (Web Speech API), screen capture, and a single "Analyze Incident" flow that calls the backend.
  • Mobile: Flutter app with the same API for description + logs.
  • Backend orchestrator: Python/FastAPI; coordinates Vision Agent (screenshot), Incident Analyst (hypotheses), Runbook Agent (pgvector RAG), and Action Agent (suggestions). Uses Google GenAI SDK with Gemini.
  • Logs service: Rust core (Aho-Corasick, high throughput) with a Python FastAPI bridge; the backend calls it for log analysis.
  • Actions service: Java/Spring Boot with an allowlist and audit log; the backend sends approved actions here.
  • Deployment: All services on Google Cloud Run; CI/CD with GitHub Actions; secrets via GCP.

Challenges we ran into

  • Keeping session state consistent when Redis was missing or invalid—we added an in-memory fallback so the app still runs.
  • A duplicate code block in the session service caused an IndentationError and broke CI/CD; we fixed it and re-validated the pipeline.
  • Integrating the Rust log parser with the Python bridge and exposing a single HTTP API for the orchestrator.

Accomplishments that we're proud of

  • Polyglot stack (Python, Rust, Java, TypeScript, Dart) with clear separation of concerns.
  • Human-in-the-loop by design: no diagnostic or remediation action runs without approval.
  • 100% test coverage targets on backend, frontend, and actions service; CI runs all suites.
  • Live on Cloud Run with a single deploy script; smoke tests verify all four services.

What we learned

We learned how to combine Gemini’s vision and text APIs with a multi-service architecture, and how critical fallbacks (e.g. in-memory when Redis is down) are for resilience. Human-in-the-loop improved both safety and trust in the agent.

What's next for SupportSight Live

  • WebSocket streaming for real-time token output from the agent.
  • Native Gemini Live API audio streaming (WebRTC) for true voice-in/voice-out.
  • Integrations with Slack and PagerDuty for incident creation and notifications.
  • A feedback loop to improve hypothesis accuracy over time.

Built With

Share this project:

Updates

Submission history