-
-
gallery image
## Inspiration
I compete on the Open Kernel Challenge leaderboard, optimizing GPU kernels that serve Meta's open models. The loop is always the same: propose a variant, validate correctness on a GPU, keep honest score, never trust an unmeasured speedup. I built KernLoop to productize that exact practitioner loop into an open source agent anyone can point at a kernel problem.
## What it does
KernLoop is an agent loop that autonomously optimizes GPU kernels. You give it a kernel problem spec (shapes, dtype, a naive baseline, a correctness tolerance). It asks a reasoning LLM, NVIDIA Nemotron, to propose a Triton kernel variant given the current champion and its measured benchmark profile. Each candidate is validated for real: a GPU checker runs it against a float32 torch reference with a relative-L2 tolerance, then times it with cold-cache discipline (L2 flush before every timed rep, warmup reps discarded, mean plus min over reps). A candidate becomes the new champion only if its mean beats the champion mean by more than 2 percent. Otherwise it is rejected, and the rejection reason goes back into the prompt. The loop repeats within a fixed budget and writes an
append-only JSONL run log plus a self-contained HTML trajectory page a human can inspect: live trajectory viewer.
## How I built it
`orchestrator.py` runs the loop. `nemotron_client.py` is an OpenAI-compatible client (a base URL plus a model name, so any Nemotron variant drops in via an environment variable) whose prompt carries the spec, the champion kernel, the benchmark profile, and the recent rejection history, under an explicit code-only output contract: the model may change block sizes, num_warps and num_stages, and split-K structure, and may not introduce host syncs, data-dependent branching, or contract changes. `nebius_jobs.py` implements the Nebius AI Cloud Serverless Jobs backend: candidate file and GPU checker are injected into the job, the checker prints one parseable result line, and every failure mode (job failed, no result line, timeout) maps to a clean logged error. `research.py` calls the Tavily search API at run start and after 3 consecutive rejections, injecting fresh Triton optimization notes into the prompt when the proposer is stuck. 70 unit tests cover the loop, promotion rule, clients, and log pipeline, no GPU required.
## What is real vs simulated, stated plainly
The LLM proposals in the demo video are real, from NVIDIA Nemotron 3.5 Lightning 30B A3B via the NVIDIA API. The Tavily research calls are real API calls, visible in the terminal. The GPU execution in the recorded run uses a deterministic synthetic backend, clearly labeled in every log line, because Nebius account billing could not be activated from my region during the hackathon window (Stripe rejected every card I tried). The Nebius path, Token Factory endpoint plus Serverless Jobs, is fully implemented in the repo and documented in the README, ready to run the moment billing works, and every run log records which backend and which model produced each number, so real and simulated results are never mixed.
## Challenges
The hard part was not code generation, it was honest measurement. A wrong answer that is fast is worthless, so correctness checking runs against a float32 reference with a rel-L2 tolerance, and promotion requires beating the champion mean by more than 2 percent, which filters benchmark noise. Keeping the Nemotron proposer inside a strict output contract took explicit prompt structure and parse-failure recovery (the video shows a rejected iteration where the model wrapped code in prose). The second challenge was retrieval on Serverless Jobs: job logs are only reachable via console and CLI, so result retrieval shells out to `nebius ai job logs` for now (marked with TODO(nebius) in the code).
## What's next
A live Nemotron-on-Token-Factory run the moment billing activates, then more workloads (attention, decode-stage GEMMs) and a rejection-reason clusterer to steer exploration.
Log in or sign up for Devpost to join the conversation.