CutGuard AI

CutGuard AI: Autonomous SRE that turns a $140,000/minute broadcast disaster into a 30-second self-healing loop.

Comment

Inspiration

In live digital entertainment, downtime is catastrophic. During high-stakes global broadcasts—such as championship esports tournaments, World Cup matches, or the Super Bowl—streaming dropouts cost major platforms up to $140,000 every single minute in lost ad impressions, brand damage, and viewer churn.

Behind large-scale cloud media pipelines, incident remediation remains slow and manual. When an FFmpeg transcoding worker crashes on an edge node at 3 AM due to a malformed frame header or codec mismatch, the typical industry Mean Time to Remediation (MTTR) averages 45 minutes.

The Cost of an Outage:
Financial Impact = MTTR × Downtime Cost per Minute
45 minutes × $140,000 / minute = $6.3 Million

We built CutGuard AI to answer a simple question: What if cloud video infrastructure could detect, triage, test, and heal production crashes autonomously in under 30 seconds—without sacrificing human engineering governance?

What it does

CutGuard AI is an autonomous Site Reliability Engineering (SRE) agent engineered for high-concurrency cloud media pipelines. It listens to live infrastructure telemetry and closes the remediation loop autonomously:

  • Passive Telemetry Ingestion: Intercepts fatal FFmpeg worker crashes (SIGSEGV, SIGABRT 137, OOM) in real time via runtime telemetry webhooks without waiting for manual alerts.
  • Sub-Millisecond Deterministic Triage: Decompiles stack traces via compiled regex in under 1 ms, isolating the failing module and line number without wasting LLM inference latency.
  • AST Blast Radius Mapping: Traverses dependency call graphs to calculate a quantitative blast-radius impact score across downstream transcoding workers and encoding profiles.
  • Cognitive Patch Synthesis: Employs Google Gemini to analyze runtime crash context and source code semantics, dynamically authoring a surgical unified Git diff.
  • Hermetic Sandbox Validation: Mounts the synthesized patch into an isolated Jest test runner, enforcing a strict 100% pass rate on unit and regression suites before code moves forward.
  • Human-in-the-Loop Governance: Halts execution using native LangGraph state interrupts, allowing an SRE to inspect the verified diff and blast-radius metrics before authorizing production deployment.
  • Instant Hot-Patching & GitOps: Injects an in-memory control-plane patch to restore live video playback in ~50 ms, while asynchronously opening a production hotfix PR and generating a structured Post-Mortem RCA.

How we built it

CutGuard AI is built around an Autonomous Agentic SRE Architecture powered by LangGraph, Google Gemini, and automated tool-calling:

  • Autonomous SRE Agent Core (LangGraph): The heart of the system is cutguard_agent—a compiled LangGraph cyclic state machine. The agent operates autonomously across 6 dedicated stages, maintaining execution state via an in-memory checkpointer and making dynamic routing decisions based on runtime feedback.
  • Cognitive Reasoning & Patch Synthesis (Google Gemini): The agent uses Gemini as its reasoning engine. When an incident occurs, the agent ingests the raw crash context, reasons over the source code semantics, and autonomously synthesizes a surgical, unified Git patch.
  • Agentic Self-Correction & Reflection Loop: Instead of blindly executing code, the agent possesses self-healing reflection. It runs candidate diffs inside an isolated Jest sandbox tool; if test assertions fail, the agent captures the compiler output, reflects on what went wrong, and autonomously re-prompts itself (up to 3 retries) until it produces a 100% verified fix.
  • Deterministic Agent Tools: The agent is equipped with specialized deterministic tools to minimize hallucination:
    • AST Tool: Statically parses dependency call trees to map downstream worker blast radius (0–100 score).
    • Sandbox Tool: Executes zero-mutation ephemeral test runs without corrupting the live codebase.
    • GitOps Tool: Autonomously stages commits, creates Git tags, and opens GitHub Pull Requests.
  • Human-in-the-Loop (HITL) Agent Governance: Built with native LangGraph interrupt() gates. The agent pauses its execution thread before touching production, allowing human operators to review its synthesized patch, test results, and blast scores on the Mission Control Console before authorizing deployment.
  • Resilient Multi-Key Rotation Engine: To prevent the agent from getting blocked during high-severity incidents, a custom key-pool manager executes automated 429 quota failovers and thermal cooldowns across multiple Gemini API accounts.
  • Live Mission Control Dashboard (Next.js & Vercel): A real-time command console connected to the Python agent via WebSockets (for instant agent node progression) and Server-Sent Events (for cluster telemetry), featuring live AST threat radars and side-by-side Git diff previews.
  • High-Speed Media Runtime (Render): The FFmpeg/Node.js transcoding cluster and Python FastAPI agent engine run as decoupled services on Render, featuring an in-memory control-plane patcher that restores live playback in <50 ms.

Challenges we ran into

  • Balancing Generative Reasoning with Deterministic Latency: Piping raw crash logs directly into an LLM introduced 3–5 seconds of unnecessary latency just to find file names and line numbers. We moved stack-trace parsing to compiled regex (<1 ms), reserving Gemini’s inference budget strictly for root-cause analysis and diff synthesis. This kept the entire remediation loop well under our 30-second target:
  Total Latency =   t_regex   +   t_AST   +  t_Gemini  +  t_Sandbox  ≈ 7.35 s ≪ 30 s
                   ╰───┬───╯     ╰──┬──╯    ╰───┬────╯   ╰────┬────╯
                    <0.001 s     0.05 s      3.8 s        3.5 s
  • Handling Upstream Model Surges (429 & 503 Errors): Frequent chaos simulations triggered Gemini API rate limits and transient capacity spikes, resulting in empty or malformed diffs. We built an automated multi-key rotation manager with thermal cooldowns and exponential backoff, coupled with defensive diff-validation guardrails to prevent unverified patches from crashing downstream tests.
  • Preventing Working-Tree Contamination in Cyclic Loops: If an agent applies a faulty patch directly to the repository during Attempt 1, the dirty state corrupts all subsequent retry attempts. We resolved this by mounting candidate patches into a zero-mutation ephemeral Jest sandbox, ensuring the live repository remains untouched until a patch passes 100% of unit tests and receives human approval.
  • Instant Live Demo Recovery vs. 4-Minute Cloud Redeploys on Render: When presenting to judges, waiting 3 to 5 minutes for Render to re-clone, build, and reboot the service after every patch would completely stall the evaluation. To deliver an instant, seamless demo without making judges wait, we decoupled runtime recovery from permanent code updates:
    • Phase 1 (Instant Judge Demo Recovery): Hot-reloads the patch into worker memory via Node require.cache invalidation in <50 ms, so judges can watch the corrupted video stream instantly unfreeze and recover in real time right before their eyes.
    • Phase 2 (GitHub PR & RCA): Asynchronously opens a hotfix Pull Request on GitHub and compiles an Enterprise Post-Mortem RCA, proving code permanence without slowing down the live presentation.

Accomplishments that we're proud of

  • Autonomous Cyclic Self-Healing Loop: Built a multi-turn LangGraph reflection loop that never trusts LLM output blindly—it executes candidate patches in an isolated Jest sandbox and iteratively feeds test failures back to Gemini until achieving a verified 100% test pass rate.
  • Deterministic Sub-Millisecond Triage: Replaced slow, token-heavy LLM log parsing with compiled regex, slashing crash triage to <1 ms and keeping the entire end-to-end remediation cycle under 8 seconds.
  • Static AST Blast Radius Engine: Developed a deterministic dependency analyzer that traces exported symbol call trees to calculate a quantitative impact score (0–100) across downstream workers and bitrate ladders without LLM hallucination.
  • Native Human-in-the-Loop Governance: Leveraged LangGraph checkpoint interrupts so the agent safely pauses state before touching production, giving SREs full visibility to review the AST threat radar and side-by-side Git diffs on the real-time Next.js dashboard.
  • Zero-Downtime Multi-Key Failover: Built an automated API key rotation manager that shields the agent from Gemini 429 rate limits and transient 503 capacity spikes through dynamic pooling and thermal cooldowns.

What we learned

  • Don't Use LLMs for Structured Data: Using Gemini to parse stack traces added 3–5 seconds of latency. We learned that pairing deterministic tools (compiled regex in <1 ms and static AST parsers) with generative reasoning (Gemini for semantic diff synthesis) creates a vastly faster, cheaper, and more reliable agent.
  • Single-Shot Code Generation Is Unsafe: LLMs frequently produce subtle syntax or logic bugs on the first pass. True autonomous reliability requires a cyclic feedback loop—coupling the agent with an isolated Jest sandbox so it can execute its own code, read compiler errors, and self-heal before touching production.
  • Human-in-the-Loop Is Essential for Enterprise Trust: Autonomous agents shouldn't have blind deploy access to production. Using LangGraph state interrupts taught us how to pause an execution thread in memory, allowing human SREs to inspect the diff and blast score while preserving full state for one-click resumption.
  • Live Video Demands Runtime Decoupling: Waiting 3 to 5 minutes for cloud services to rebuild and reboot on Render kills a live stream and stalls a live judge presentation. Decoupling immediate runtime survival (in-memory require.cache invalidation in <50 ms) from permanent GitOps commits is the ideal pattern for zero-downtime systems.
  • Defensive API Resilience Is Mandatory: Upstream 429 Too Many Requests and 503 Unavailable spikes are inevitable during repetitive chaos testing. Building a multi-key rotation pool with thermal backoffs taught us that production agents must be architected to survive upstream quota outages without crashing.

What's next for CutGuard AI

  • Kernel-Level eBPF Telemetry: Move from application-level crash interception to native Linux eBPF probes, detecting FFmpeg segfaults, packet drops, and memory leaks directly at the kernel layer before processes crash.
  • Predictive Pre-Crash Remediation: Train lightweight anomaly models on streaming telemetry (transcode duration, frame jitter, memory slope) to synthesize and deploy defensive patches before an Out-of-Memory (OOM) termination occurs.
  • VMAF/SSIM Video Quality Guardrails: Integrate automated perceptual video quality scoring into the Jest sandbox runner, ensuring synthesized patches not only fix crashes but preserve 100% visual fidelity without bitrate degradation.
  • Progressive Canary Traffic Shifting: Expand deployment to route 5% of live ingest chunks to patched workers first, verifying real-time latency and error rates before completing full cluster rollouts.
  • Slack & PagerDuty Interactive Gates: Extend the LangGraph Human-in-the-Loop gate to Slack and PagerDuty webhooks, enabling on-call SREs to review AST blast scores and approve production hot-patches with a single tap on mobile.

Built With

Share this project:

Updates

Submission history