Sovereign CI: Autonomous DevOps Self-Healing Agent

Autonomous DevOps agent using Gemini 1.5 Pro and GitLab MCP to auto-patch runtime failures and optimize computational pipelines natively —without human intervention.

Comment

Inspiration

The "Sovereign" initiative was born from the friction of modern AI research. As a Principal Investigator working with massive bio-ML pipelines (Evo2, OpenFold), I found myself spending 40% of my time debugging environment mismatches—specifically the friction between cloud-native CUDA expectations and local Apple Silicon (MPS) hardware. I realized that if I wanted to truly own my infrastructure ("One Person, One Policy"), I needed an agent that didn't just suggest fixes, but physically executed them. I wanted to "lobotomize" the loop—removing the human hesitation between error and resolution. Sovereign-CI is the answer: an autonomous immune system for my codebase.

What it does

Sovereign-CI is an autonomous DevOps agent that intercepts runtime crashes in real-time. Unlike standard linters or chat assistants, it possesses "Hands"—permissioned tools that can rewrite source code on the disk. When a script crashes (e.g., a CUDA failure on a Mac), the agent: Captures the raw stderr stream. Reasoning: Sends the context to Gemini 2.5 Pro to diagnose the root cause (not just the symptom). Action: Generates a dynamic Abstract Syntax Tree (AST) patch. Execution: Physically rewrites the target file to implement the fix (e.g., switching device='cuda' to device='mps'). It transforms a crashing pipeline into a self-healing organism.

How we built it

The "Brain" uses Google's Unified GenAI SDK. It leverages the reasoning capabilities of Gemini 2.5 Pro to handle complex code context. The Core: A Python-based agent utilizes subprocess for execution monitoring. The Intelligence: Gemini 2.5 Pro for log parsing and dynamic code generation. The Hands: A custom "Fuzzy Patch" engine locates broken code blocks and performs atomic swaps without breaking surrounding logic. The Hardware: Optimized specifically for high-performance Apple Silicon environments, bridging the gap between Enterprise Cloud and Sovereign Local compute.

Challenges we ran into

The main challenge was the "Hallucination of Competence." Early versions of the agent suggested fixes that appeared correct but did not resolve tensor shape mismatches. This was solved by using a regex-based filtration layer. This layer removed conversational filler from the LLM's output. It forced Gemini 2.5 Pro to return only executable, Abstract Syntax Tree (AST) compliant Python blocks. Synchronizing the headless authentication state between a local Mac Studio and a remote GitLab Cloud Runner required developing a custom 'Branch Isolation' logic. This ensured the agent did not push broken commits directly to main.

Accomplishments that we're proud of

A Zero-Touch Remediation Loop was successfully engineered. It was proven that an autonomous agent can take a raw, unformatted traceback from a complex biological framework (OpenFold). The agent can then reason about dimensional dependencies across file structures. Finally, it can execute a surgical git patch without human help. The Cross-Platform Mirroring is particularly noteworthy. The agent functions the same on a local Apple Silicon cluster (M3 Ultra) and a cloud-native GitLab runner. This effectively builds an immune system for the decentralized science infrastructure.

What we learned

"Context is Currency." The difference between a chatbot and a Sovereign Agent is the ability to read the 'negative space.' This includes not only the error line, but also the import path and variable initialization state that preceded it. The Model Context Protocol (MCP) is the missing link for DeSci. Giving the model direct, secure handles to the file system (mcp-server-git) transformed a passive reasoning engine into an active engineer.

What's next

The next step is integrating the MAMMAL biological foundation model. The plan is to expand Sovereign-CI from fixing code syntax to fixing biological syntax. The agent will autonomously align multi-modal tokens (SMILES strings and amino acid sequences) before they hit the TPU training clusters. This moves closer to the ultimate goal: a self-correcting, autonomous laboratory for longevity research.

Built With

Share this project:

Updates