Cryptographically Signed Agent Loop for GPU Kernels

Ran on one NVIDIA RTX 3060 Ti, consumer desktop card with 8 GB.

11.87× geometric-mean speedup over the twelve of fourteen shapes that have a runnable official baseline — from 2.37× on the shape already near its arithmetic limit to 31.51× on the longest sequence.
Shape 14 reaches 88.7% of what the card can physically do: 100,000 tokens at batch 32, a shape the official code cannot run at all on this hardware.
We beat PyTorch's own sdpa on every shape, by 2.3× to 20.9× — and on the two extreme shapes it cannot run at all while ours does.
The one shape where we had to write our own reference, we held to twice the competition's strictness. Shape 14 is too large for TikTok's baseline to run at all, so we wrote a streamed reference and validated it against their real implementation at 1,024, 2,048 and 4,096 tokens at 1e-3 absolute — twice as strict as the competition's 2e-3.
94 correctness trials, 17,370,759,168 elements compared, zero violations.
Mean utilisation is 42.7% with every shape weighted equally. The organiser has not published the weighting, and the same fourteen measurements give 35.5% to 88.6% depending on which rule is used — we published all five and optimised against the low end.

This project combines modern kernel-agent techniques with cryptographic signing and loop engineering, so the agent doing the optimising stays on task and cannot fake its own results. Every number came out of a locked program the agent cannot edit, using a single-use permit from a run budget only a human can sign.

More information: GitHub Repository

Full technical breakdown: TECHNICAL_BREAKDOWN.md

Inspiration

We tried both track 2 and track 3 at the same time, with an AI agent optimising in a loop. It produced good numbers but we threw all of them away, because the agent had been allowed to run its own measurements and we could not prove it hadn't drifted into measuring something easier than the real task.

That turned out to be a known problem rather than our own incompetence. CUDA-L1's postmortem found that a third of its reinforcement-learned "solutions" timed work on a side stream the clock never watched. Sakana's optimizer edited its own evaluator. A score-seeking agent attacks the score when the score is easier to attack than the task, and neither of those teams asked for that behaviour.

The most useful thing we read came from the sponsor's own research group: CUDA Agent (ByteDance Seed and Tsinghua). Their system reaches 2.11× geomean over torch.compile, and the part that mattered to us was not the kernel. It was that they had independently arrived at the same conclusion. Protected profiling scripts the agent cannot modify. Forbidden fallback calls. Measurement it cannot reach. If the people who do this professionally had to build those guardrails for a reinforcement-learning system, we were going to need them too.

What it does

It makes TikTok's transformer benchmark run faster on all fourteen test shapes, and it proves the numbers weren't faked.

An AI agent writes Triton and CUDA kernels. A separate, cryptographically locked program measures them. The agent can read that program but cannot change it, and cannot run a single measurement without a permit signed by a human. Every result carries its own evidence: 300 raw timing samples, every correctness trial, and the hash of the exact file that was measured.

We achieved a geometric-mean of 11.87× over the twelve shapes with a runnable baseline, on one consumer RTX 3060 Ti. All fourteen pass the precision test. We stuck to python, but CUDA written in C++ was also tested upon.

How we built it

The referee came first, and the kernels second. That order is the whole project.

Twenty-nine files: the official benchmark, the harness, the shape definitions, the tools, are all hashed, and the list of hashes is signed with a private key the agent has never had. Before every single run the controller re-hashes all twenty-nine and refuses everything on one mismatch.

To run anything at all, the agent needs a permit. A human signs one capability worth N runs, and the system spends it one permit at a time, each permit welded to the sha256 of one specific candidate file, each usable once. Over the campaign that came to 272 permits issued and 271 consumed. The run itself happens in a sandbox with no network, no home directory and the source mounted read-only.

Optimising was gated too. Before each attempt the agent had to cite at least two existing research notes and carry the current hash of the index, which proves it actually read them that cycle. Then a plan with a hypothesis, a numeric prediction and citations written as file:line. The gate then looks each one up and copies the quoted text into the log, so a made-up citation does not get through. Then exactly one run, after which the gate shuts. Three attempts with no improvement closes off that approach and forces a written postmortem before anything else can start.

Only then, the kernels are made, with eight authored Triton kernels and a dispatcher, with the whole multi-layer forward pass captured as one CUDA graph.

Challenges we ran into

Shape 14 does not fit anywhere. 100,000 tokens at batch 32 needs roughly 160 TB of attention matrix. TikTok's baseline cannot run it on any GPU, so there was nothing to be correct against. We wrote a streamed reference, then validated that against their real implementation at 1,024, 2,048 and 4,096 tokens where theirs still fits, at 1e-3 absolute, twice as strict as the competition's requirement. It now runs in 48.271 s at 88.7% of the card's physical maximum, executed as 32 serial batch-1 calls, since the whole batch doesn't fit in 8 GB at once.

We also ran out of time on CUDA. k018, k023 and k024 are hand-written CUDA kernels, but none beat the Triton versions before the deadline, so none of them ships. We were still finding wins when the clock ran out, and the last structural change landed hours before the submission.

Accomplishments that we're proud of

We attacked our own referee and it caught us. Project/harness/redteam/ holds two working cheats. One sabotages the baseline's maths after the hash check, so any candidate would "match" a broken reference. The other returns a cached output whenever the input's memory address repeats, which is exactly what a benchmark timing loop does. Both were run through the harness. Both were caught, and the catches are in the journal.

Every row also comes from one file. An earlier board of ours drew its rows from four different builds and reported a single average, describing a program that never existed. We threw it out and re-measured everything on the file we actually submit.

What we learned

Having a research base is not the same as using it. Every plan cited one, because the gate refused otherwise. But CUDA Agent's loop profiles first and implements second, while ours planned first and profiled only when something looked wrong. We had the better paper on disk the whole time and took less from it than we should have.

Optimisation reasons are also often wrong even when the optimisation works. We split the attention head loop into its own grid dimension for more parallelism. The parallelism did almost nothing, every shape but one already filled the card's 38 SMs. The gain came from the side effect nobody predicted: with heads written into one buffer, the output projection runs at full width instead of once per head padded to Triton's 16-wide minimum. +26.4% on the two shapes with head width 8.

What's next for Cryptographically Signed Agent Loop for GPU Kernels

The four launch-bound shapes. Shapes 2, 3, 7 and 12 sit at 5.3%, 16.2%, 17.7% and 34.1% utilisation, the lowest on the board, because the grid can't fill 38 SMs. A sequence-persistent kernel is the obvious next attempt.

Finish the CUDA path. Three hand-written CUDA kernels exist and none beat Triton before the deadline. We would like to further explore this space if given the time.