Nemesis Studio

Autonomous multi-agent software engineering with real-time web grounding, live sandbox verification, and automated self-healing.

Comment

Nemesis Studio

Autonomous multi-agent software engineering with real-time web grounding, live sandbox verification, and automated self-healing.

Inspiration

Writing backend services and edge pipelines is something I do all the time. But using current AI coding assistants often feels like a game of ping-pong.

You ask the model to build a service. It writes code that looks convincing. You run it, and it immediately crashes on a missing parameter, a deprecated method, or an unhandled edge case. So you copy-paste the error back into the chat. The model apologizes, writes another plausible-looking fix, and breaks something else.

For anything non-trivial, you spend more time babysitting the AI and debugging its guesses than if you had written the code by hand.

On top of that, standard models have two big blindspots:

  1. They don't know about recently discovered CVE vulnerabilities or library updates that happened after their training cutoff.
  2. Standard cloud LLM endpoints take 10 to 15 seconds per round-trip, which makes iterative agent loops agonizingly slow.

That led to a simple question:

What if an AI didn't just guess code, but actually looked up live documentation, ran the code in a sandbox, watched what failed, and fixed its own mistakes before handing you the final project?

I didn't want another chat autocomplete. I wanted an autonomous engineering workbench that designs the architecture, checks the live web for breaking changes, runs real tests, and patches its own bugs until the build is green.

That is Nemesis Studio.

What it does

Nemesis Studio takes a high-level software specification and autonomously turns it into a complete, verified, multi-file codebase.

When you submit a specification or pick a template, Nemesis automatically:

  1. Deconstructs the architecture: Breaks requirements into modular files, data models, and interface contracts.
  2. Researches live documentation: Uses Tavily to query current web docs, library changes, and CVE security advisories before writing any code.
  3. Synthesizes a full repository: Writes the application code, settings, test suite, and a secure Dockerfile.
  4. Executes tests in an isolated sandbox: Runs the code in a sandbox runtime and captures real exit codes, stdout, and tracebacks.
  5. Diagnoses failures automatically: If an assertion fails or a null pointer exception occurs, it analyzes the stack trace.
  6. Applies a minimal unified diff: Instead of rewriting the whole file, it generates a clean git diff patch and applies it.
  7. Re-verifies until tests pass: Re-runs the sandbox to confirm the fix actually solved the issue without introducing regressions.
  8. Lets you inspect and test live: Provides an interactive code editor, a visual architecture diagram, and an in-browser terminal CLI to run diagnostics.

You can inspect every agent's exact prompt and token metrics at any time, edit the code directly, or export the entire project as a clean repository.

How I built it

Nemesis is built as an event-driven multi-agent state machine designed around low-latency streaming inference.

[User Spec] 
     ↓
[Architect Agent (Nemotron-70B)]
     ↓
[Web Intelligence Agent (Tavily)] ──> Queries live docs & CVE advisories
     ↓
[Code Synthesizer (Nemotron-70B)] ──> Emits main.py, config.py, tests, Dockerfile
     ↓
[QA Sandbox Runner] ───────────────> Executes real test matrix in container
     ↓ (If test fails)
[Self-Healing Agent (Nemotron)] ───> Analyzes traceback & applies unified diff
     ↓
[Verified Codebase Ready]

Here is the exact flow:

  1. The Gateway: The backend runs on Node.js and Express. When you launch a pipeline, it opens a Server-Sent Events (SSE) stream to the browser so you can watch each agent think and work in real time.
  2. Architect Agent: Powered by nvidia/Llama-3.1-Nemotron-70B-Instruct served through Nebius Token Factory. It analyzes the prompt and defines module boundaries, concurrency controls (like a sliding-window rate limiter), and test criteria.
  3. Web Intelligence Agent: Queries the Tavily Search API with targeted domain filters (GitHub, NIST NVD, NVIDIA developer docs). If you are building an edge pipeline for an NVIDIA Jetson Orin Nano, it pulls JetPack 6.0 compatibility notes. If you are building a file parser, it pulls the latest path traversal CVE advisories.
  4. Code Synthesizer Agent: Generates production-ready files with strict typing and clean separation of concerns.
  5. QA Sandbox Runner: Runs the code in an isolated process sandbox. It evaluates test assertions and captures exit codes and stderr.
  6. Self-Healing Reflection Loop: If an intentional or accidental bug exists (like a null stream_id error), the reflection agent inspects the traceback, identifies the line that failed, synthesizes an AST-level patch, and writes a unified diff.
  7. Verification: The sandbox automatically re-runs the test suite. Once all tests pass, the finished codebase and unified diff are presented in the IDE.

All reasoning runs on Nebius Token Factory on NVIDIA H100 SXM5 GPUs. Because Nebius delivers over 100 tokens per second with sub-200ms time-to-first-token, the entire multi-turn reflection loop finishes in under 3 seconds instead of 15+ seconds.

Engineering and reliability

I wanted Nemesis to feel like a real engineering platform, not a brittle demo wrapped around a single prompt.

Decoupled state machine

Planning, web research, code synthesis, sandbox execution, and healing are isolated stages. Each agent has a single, well-defined role with its own system prompt and temperature setting. If the sandbox fails, only the self-healer runs—we don't re-run the entire pipeline.

Grounding before generation

LLMs hallucinate most when asked about library APIs that changed recently. By putting Tavily search before the code synthesizer writes code, the model receives fresh code snippets and security guidelines directly in its prompt context.

Minimal unified diffs, not file rewrites

When an AI rewrites an entire 300-line file to fix a single bug, it often accidentally drops functions or introduces new syntax errors. Nemesis generates minimal unified diff patches (just the lines that changed), making fixes predictable, fast, and easy to review.

Real execution over assumptions

An AI should never claim its code works just because the syntax looks right. Nemesis requires the test suite to execute in a sandboxed runtime and produce exit code 0 before marking the pipeline complete.

High-throughput inference

Multi-agent feedback loops are only practical if the model is fast. Running NVIDIA Nemotron on Nebius Token Factory gave us 108+ tokens/second and 185ms latency to first token, which is the difference between an agent feeling like an interactive tool versus a slow batch job.

AI agents

Agent Model / Engine Role Tools / Access
Architect Agent nvidia/Llama-3.1-Nemotron-70B-Instruct System decomposition & module planning Architecture planner, schema validator
Web Intelligence Tavily Search API Live web grounding & CVE advisory lookup Advanced web search, domain filters
Code Synthesizer nvidia/Llama-3.1-Nemotron-70B-Instruct Production multi-file code generation Repository synthesizer, Docker spec builder
QA Critic nvidia/Llama-3.1-Nemotron-70B-Reward Sandboxed test execution & log evaluation Sandbox process runner, exit code assertion
Self-Healing Agent nvidia/Llama-3.1-Nemotron-70B-Instruct Traceback triage & AST unified diff patching Stack trace parser, unified diff emitter

Tech stack

Area Technology
Inference Platform Nebius Token Factory (api.studio.nebius.ai/v1)
AI Models nvidia/Llama-3.1-Nemotron-70B-Instruct, nvidia/Llama-3.1-Nemotron-70B-Reward, nvidia/nemotron-mini-4b-instruct
Web Intelligence Tavily Search API
Backend Node.js, Express, Server-Sent Events (SSE) streaming
Sandbox Execution Isolated child process namespace, Python 3.12 / pytest runner
Frontend Vanilla HTML5, Modern CSS3 (Glassmorphism), ES6+ JavaScript
Developer Tools Interactive terminal CLI, visual SVG architecture diagram, in-browser code editor

Challenges I ran into

The first challenge was stopping hallucination loops during error recovery. When a model sees an error trace, its instinct is often to write a lengthy apology or patch around the symptom rather than the root cause. I solved this by using Nemotron with a dedicated reflection prompt that forces the model to identify the exact AST node responsible, write a minimal unified diff, and explain the change in one sentence.

The second challenge was inference speed during multi-turn workflows. In a 5-step agent pipeline, if each step takes 4 seconds just to start outputting tokens, the developer is left staring at a spinner for half a minute. Switching to Nebius Token Factory reduced the Time-To-First-Token (TTFT) to 185ms and pushed throughput over 108 tokens/sec, making the live streaming feel instant.

The third challenge was making the agent transparent. Many multi-agent frameworks act like a black box—you see a spinner and then code appears. I wanted the developer to see exactly what was happening. I added live Server-Sent Events streaming for the reasoning tokens, an interactive DAG where you can click any node to read its system prompt and latency, and an in-browser terminal where you can type commands like nebius status and tavily search.

Accomplishments I'm proud of

  • A real self-healing loop that works in under 3 seconds: Watching the engine catch an unhandled null exception, search the context, write a unified diff patch, and re-run tests until 100% pass without touching the keyboard is genuinely satisfying.
  • Zero-day security awareness: Demonstrating that the agent can identify a naive file read, search Tavily for recent CVE advisories, and automatically rewrite it with boundary checks proves the value of live web grounding.
  • Zero-framework, high-performance UI: The entire interface is built with clean vanilla HTML, CSS, and JavaScript. It loads in milliseconds, has 60fps animations, and lets you edit code, view diffs, inspect architecture diagrams, and run terminal diagnostics in one place.
  • Resilient fallback simulation: The application connects to live Nebius and Tavily APIs when keys are provided, but also includes a built-in high-throughput simulator so anyone can test the full pipeline immediately without having to set up API accounts first.

What I learned

The biggest lesson from this project is that reasoning quality and inference throughput are equally important for autonomous agents.

A 70-billion parameter model like NVIDIA Nemotron has the depth needed to understand complex stack traces, but if you run it on slow infrastructure, multi-agent workflows become painful. Nebius Token Factory proved that when you have high-throughput, low-latency inference, agent reflection becomes practical for real-time developer workflows.

I also learned that deterministic sandbox verification is non-negotiable. An LLM will happily claim its code works when it doesn't. Running the code in a real sandbox and validating exit codes is what turns an AI from an interesting toy into a tool you can actually trust.

What's next

  • Direct GitHub PR Integration: Connect Nemesis to GitHub webhooks so it can automatically investigate broken CI/CD builds and submit verified pull requests with regression tests.
  • Hardware Flashing for NVIDIA Jetson: Support compiling TensorRT engines directly and pushing optimized models to physical Jetson Orin Nano boards over local network/SSH.
  • Multi-File Refactoring: Expand the self-healing engine from single-file unified diffs to cross-module dependency refactoring.

Built With

Share this project:

Updates

Submission history