Project X

An intelligent multi-tier agent built with NVIDIA Nemotron on Nebius

Comment

Inspiration

In modern DevOps and site reliability engineering (SRE), automated CI/CD dependency bumps and security advisories often cause unexpected runtime crashes. When fundamental libraries undergo major architectural shifts (e.g., Pydantic v1 to v2 removing core decorators like @validator), production microservices fail with cryptic stack traces.

Developers and on-call engineers typically spend hours reading through dispersed changelogs, outdated StackOverflow posts, and breaking-change matrices to find non-breaking fixes.

We asked ourselves: What if an autonomous agent could not only diagnose the root cause in milliseconds, but also fetch real-time documentation and generate verified, non-breaking unified Git diff patches autonomously? That vision became Project X.

What It Does

Project X is an autonomous incident triage and vulnerability remediation agent designed for DevOps and SRE workflows.

  1. AST Triage & Log Decomposition: The agent accepts raw error stack traces alongside vulnerable source code snippets.
  2. Dynamic Multi-Tier Routing: It routes parsing to a fast, lightweight tier before escalating complex code generation to a high-reasoning tier, optimizing both token latency and compute costs.
  3. Real-Time Web Grounding: It uses Tavily AI Search to query live official documentation, CVE repositories, and migration guides in real time to avoid LLM hallucinations.
  4. Verified Patch Synthesis: It streams a production-ready, syntax-highlighted unified Git diff (+ / -) that can be exported or copied with one click.
  5. Live Agent Inspector & Telemetry HUD: Displays the real-time reasoning timeline, latency timers, token velocities (tokens/sec), and estimated compute cost savings (~84.6%).

How We Built It

We built Project X with a lean, production-ready stack designed for rapid local evaluation and containerized deployment:

  • Backend Engine: Python 3.11+ and FastAPI with asynchronous streaming via Server-Sent Events (SSE).
  • AI Orchestration & Routing:
    • Fast Triage Tier: nvidia/nemotron-mini-4b-instruct hosted on Nebius Token Factory for ultra-fast (<400ms) log parsing, stack trace deconstruction, and tool query generation.
    • Deep Reasoning Tier: nvidia/Llama-3.1-Nemotron-70B-Instruct on Nebius Token Factory for complex code synthesis and non-breaking AST diff patching.
  • Ground Truth Retrieval: Tavily AI Search API for grounding the reasoning process with authoritative migration guides and official documentation.
  • Frontend Dashboard: React 18 (Vite) and Tailwind CSS with a dark-mode DevOps HUD, an interactive Git diff viewer, and a real-time Agent Inspector.
  • Local Persistence & Portability: Embedded SQLite for logging incident traces and a complete docker-compose.yml for single-command evaluation.

Nebius Token Factory & NVIDIA Nemotron Integration

Rather than passing every request to a massive, expensive LLM, Project X introduces Dynamic Multi-Tier Cognitive Routing powered entirely by Nebius Token Factory:

Stage Model Selected Role Latency
Stage 1: Log Triage nvidia/nemotron-mini-4b-instruct AST decomposition & search query formulation ~380ms
Stage 2: Search Grounding Tavily AI Search API Live CVE & migration doc retrieval ~450ms
Stage 3: Patch Synthesis nvidia/Llama-3.1-Nemotron-70B-Instruct High-cognitive diff generation & code verification ~3.35s

By using Nemotron Mini (4B) for preliminary log parsing and escalating exclusively to Nemotron 70B for the final patch synthesis, Project X reduces overall compute consumption and token costs by ~84.6% compared to single-tier 70B architectures.

Real-Time Grounding with Tavily AI Search

LLMs operating on code remediation frequently hallucinate deprecated syntax or reference outdated blog posts. To guarantee deterministic accuracy:

  • When an incident is parsed, the fast tier formulates targeted technical queries (e.g., Pydantic v2 migration guide field_validator model_dump BaseSettings).
  • Tavily AI Search retrieves real-time official documentation snippets directly from authoritative sources (such as docs.pydantic.dev).
  • The retrieved context is injected into Nemotron 70B's reasoning prompt, ensuring every generated unified diff conforms strictly to the latest API standards.

Challenges We Ran Into

  1. Streaming Multi-Tier Telemetry: Synchronizing SSE events across two distinct model tiers and external search tools while rendering live telemetry (latency, token rate, tier badges) in React required careful event schema design.
  2. International Billing & Verification Hurdles: As undergraduate participants in India, navigating international bank verification holds for cloud platforms was a challenge. We built an authentic, deterministic fallback mode into the backend client so the entire system, telemetry stream, and inspector HUD remain testable offline without third-party API dependencies.
  3. Preventing Code Hallucinations: Large models often rewrite entire files rather than outputting minimal, surgical changes. Engineering structured prompts to force unified Git diff standards (--- a/, +++ b/, @@) ensured minimal, safe patch application.

Accomplishments That We're Proud Of

  • Zero-Lag Developer Experience: The Agent Inspector HUD streams tokens at over 130 tokens/sec with sub-second initial triage.
  • Accurate Hero Scenario Execution: Fully resolved the real-world Pydantic v1 to v2 migration error (converting @validator to @field_validator and updating imports to pydantic_settings) with zero manual code modifications.
  • Clean Open-Source Scaffolding: Delivered a modular monorepo with an MIT License, comprehensive documentation, and single-command Docker deployment.

What We Learned

  • Specialized Routing Beats Brute Force: Smaller models like nemotron-mini-4b-instruct excel at deterministic extraction tasks and run with significantly lower latency than 70B models. Combining small and large models yields better responsiveness and lower operational costs.
  • Grounded RAG is Essential for DevOps: Having a search agent like Tavily verify breaking changes against live documentation eliminates the risk of proposing deprecated APIs.

Limitations & Edge Cases

In the spirit of engineering rigor:

  • Deterministic Focus: Project X is currently optimized for deterministic runtime dependency conflicts, library deprecations, and syntax errors.
  • Stateful & Schema Migrations: It does not yet automatically execute stateful database migrations (e.g., Alembic migrations) or distributed network topology reconfigurations, which still require human oversight.
  • Sandbox Verification: Patch verification currently relies on static AST validation; future iterations will include isolated Docker sandbox test executions.

What's Next for Project X

  • Automated GitHub PR Creation: Enabling the agent to automatically fork, branch, commit the patch, and open a pull request directly on the affected repository.
  • Ephemeral Test Sandboxing: Spinning up isolated Firecracker microVMs to run test suites against the generated patch before human review.
  • CLI & Slack Integration: Building a CLI tool (project-x triage) and a Slack incident bot for seamless SRE alerting workflows.

Built With

Share this project:

Updates

Submission history