MLGenie

Full 6 Stage CI/CD Pipeline
Scalable Deployment of ML Model to Google Cloud Vertex AI

Inspiration

One of the biggest bottlenecks in the Machine Learning Software Development Lifecycle is not building a model, but turning prototype ML code into a secure, reliable, production-ready workflow. We experienced this challenge firsthand while working with researchers & data scientists in Machine Learning labs, where strong experimental models frequently stalled before reaching production use. In traditional SDLC, teams already deal with testing, security, and deployment, but the ML lifecycle introduces additional complexity such as data validation, model training orchestration, quality evaluation, and scalable cloud deployment. These steps are critical for real-world ML systems, yet they are often manual, fragmented, and require significant engineering effort.

This personal experience inspired us to build MLGenie – a custom GitLab Duo Agent that brings Gitlab's best DevSecOps principles into the ML lifecycle by automatically configuring a structured 6-stage ML CI/CD pipeline with security scanning, validation, testing, training, evaluation, and deployment to Google Cloud Vertex AI. Our goal is to give GitLab developers an ML engineering teammate that can help them move from prototype to production with confidence, so teams can focus more on innovating rather than navigating complex MLOps setup.

What it does

MLGenie is an MLOps expert agent that helps developers transform an experimental Scikit-learn machine learning project into a secure, production-grade MLOps CI/CD pipeline directly within their GitLab development workflow.

Consider a real-world scenario where a team has built a classification model to predict hotel cancellations for a travel company. The model performs well in a notebook or local script, but deploying it into a production system requires significantly more engineering work. The project now needs secure CI/CD automation, dataset validation, reproducible training pipelines, model quality enforcement, artifact versioning, and a reliable deployment path to a live cloud endpoint. This transition from experimentation to production is where many ML projects face friction.

MLGenie addresses this gap by acting as a collaborative ML engineering teammate. Developers can begin interacting with the agent directly from their IDE without switching tools. The agent first analyzes the repository context – inspecting dependencies, training scripts, configuration files, and pipeline definitions – to understand the machine learning task, data flow, and deployment goals. Using this contextual understanding, MLGenie incrementally helps agentically take action to configure the MLOps pipeline, guiding developers through each stage of productionization instead of requiring them to manually design and implement complex MLOps infrastructure.

ML Pipeline Stages Configured by MLGenie

MLGenie agentically carries out concrete productionization tasks using GitLab Duo platform tools rather than only providing guidance to developers. It uses repository inspection tools to understand project structure, file creation and editing tools to configure .gitlab-ci.yml and supporting pipeline modules, execution tools to run validations and tests, and merge-request tools to package infrastructure updates into structured, reviewable workflow changes. This allows the agent to configure security scanning, data validation, testing, training orchestration, evaluation gates, and deployment workflows as part of a complete ML CI/CD pipeline, making setup more transparent, reviewable, and aligned with real engineering collaboration practices.

Stage	Purpose	How MLGenie Agent Configures It
Security	Secure the ML pipeline before training begins	Adds dependency scanning, secret detection, SAST templates, and `pip-audit` checks to detect vulnerabilities early in CI.
Validate	Ensure dataset integrity and schema correctness	Generates schema-aware validation modules that verify data structure, null thresholds, class balance, and constraints from Google Cloud Storage datasets.
Test	Improve reliability of ML utilities and pipeline logic	Configures unit tests with JUnit reporting to validate preprocessing, evaluation functions, and pipeline behavior within GitLab CI.
Train	Enable reproducible and versioned model training	Sets up training workflows for Scikit-learn models, produces artifacts such as `model.pkl`, metrics and metadata files, and uploads versioned outputs to Google Cloud Storage.
Evaluate	Prevent low-quality models from reaching production	Adds automated quality gates for performance metrics such as recall, precision, and ROC-AUC before deployment is allowed.
Deploy	Operationalize models as live cloud services	Configures deployment to Google Cloud Vertex AI, registers model versions, deploys endpoints using the Vertex AI Scikit-learn prediction container, and runs endpoint health checks.

MLGenie CI/CD Pipeline Diagram

Figure 1. MLGenie configuring a secure 6-stage Machine Learning CI/CD pipeline integrated with Google Cloud Vertex AI.

Google Cloud & Vertex AI Integration

One of Google Cloud’s strongest services for operationalizing AI/ML models is Vertex AI, so we wanted to make it a core part of MLGenie’s pipeline configuration workflow. We designed the agent to help developers move from a trained Scikit-learn model artifact to a real Vertex AI endpoint, with pipeline stages that version artifacts in Google Cloud Storage, prepare deployment, and support managed online inference. This was a strong fit for our use case because Vertex AI provides prebuilt serving containers for Scikit-learn, which means developers do not have to build and maintain their own custom serving container just to deploy a model.

We also set up the GitLab-to-Google-Cloud integration using Workload Identity Federation (WIF), which allows CI/CD jobs to authenticate securely to GCP. This made the workflow both more secure and more practical, while giving developers an easier path to deploy trained models through a GitLab-native pipeline onto Vertex AI.

Custom MCP Server for Generating Explainer Videos

We wanted to explore how AI agents can contribute beyond pipeline automation and deployment, so we extended MLGenie with a creative agentic capability for generating multimedia project documentation in the form of animated explainer videos. We configured and connected our Duo Agent to a custom MCP animation server that allows the agent to generate explainer videos describing ML pipelines, model workflows, and system architecture. This feature was designed to help teams make ML projects more interpretable, shareable, and engaging, especially for onboarding new engineers or communicating project structure across teams. The videos provide a visually intuitive way to document ML systems, making technical workflows easier to understand while also offering a more fun format for internal knowledge sharing.

The MCP animation server is built on Manim (Mathematical Animation Engine) – an open-source Python framework used for programmatically creating technical animations and visualizations. The MLGenie agent can send animation instructions as code to the MCP server, which renders dynamic, animated videos illustrating pipeline stages, data flow, and deployment progression. 🔗 You can find the MLGenie agent configuration and MCP setup instructions here: MLGenie GitLab Repository

MLGenie Manim Explainer Demo

Figure 2. Animated explainer video created by the Duo Agent through MCP.

What we learned

Building MLGenie taught us that productionizing ML is not just a deployment problem – it is a workflow design problem that creates real complexity for engineers. Dependency security, data validation, evaluation gates, artifact versioning, and deployment all need to work as one connected pipeline. From an architecture perspective, this is why we intentionally built MLGenie as a custom agent rather than a fixed flow. Instead of acting as an automated "blackbox", the agent moves stage by stage, summarizes what it configured, and keeps the developer in control through a Human-in-the-Loop workflow. That was important to us because production ML infrastructure is too critical to hide behind one-click automation. We realized that it was important for our agent to reduce complexity while still keeping the developer in charge of approving and customizing each part of the workflow as required.

What's next for MLGenie

We hope to keep iterating on MLGenie within the GitLab Duo Agent Platform, with the goal of making it even more useful and accessible to developers working on real machine learning projects. Right now, the agent is focused on Scikit-learn pipelines, but we want to expand its expertise to support a deeper ML stack, including frameworks such as PyTorch, TensorFlow, and Hugging Face.

We also want to make MLGenie more end-to-end by extending it beyond initial pipeline setup. One exciting direction we may explore next is building a complementary flow for pipeline failure diagnosis & observability, so teams can not only configure production pipelines but agentically resolve issues. We hope that MLGenie can continue to grow into a valuable agent for GitLab developers bringing machine learning systems into production!