Inspiration
Review bots comment on diffs without running the project's tests, so their findings cannot be trusted. This agent only keeps findings backed by test evidence, and says so when a finding has none.
What it does
Nemotron Review Agent is a code review agent for the Coding and Agentic Engineering track. Given a unified diff, it runs a three-step loop:
- Review: an NVIDIA Nemotron model reviews the diff and returns structured findings as JSON (file, line, severity, category, fix suggestion).
- Test: the agent runs the project's test suite (auto-detects pytest, npm, or go test, or takes an explicit command).
- Verify: the model re-examines each finding against the test output. Findings confirmed by failing tests are kept with the test evidence cited. Findings with no test coverage are dropped or marked unverified.
The repository includes a runnable example (examples/demo_shop) where two of five tests fail on purpose, so the keep/drop behavior is observable end to end. A Gradio web UI (app.py) is included for the hosted demo.
How we built it
Python CLI plus a Gradio web UI. All model calls go through Nebius Token Factory's OpenAI-compatible API. The default model is nvidia/nemotron-3-super-120b-a12b, switchable to nvidia/nemotron-3-nano-30b-a3b for cheap runs or nvidia/nemotron-3-ultra-550b-a55b for deep reasoning via NEMOTRON_MODEL. Structured output uses JSON mode.
Challenges we ran into
- Hugging Face's free CPU Spaces require a PRO subscription, so the hosted demo runs on the free ZeroGPU tier, which forced a @spaces.GPU-decorated function in the app.
- Token Factory's $1 trial credit did not show in the console balance immediately after signup, which made the first hour of debugging confusing.
What we learned
- Token Factory is OpenAI-compatible as documented. client.models.list() returned the Nemotron IDs from the docs. No SDK changes needed.
- Structured output (JSON mode) worked on nvidia/nemotron-3-super-120b-a12b. Six model calls across three end-to-end runs, valid JSON every time.
- Roughly 5 to 10 seconds per call. Fast enough for an interactive review loop.
Note: the hosted demo is currently paused because I revoked the API key to prevent abuse. The demo video is a genuine recording of a live run.
