What we built
EvidenceLoop is an experimental coding agent for one synthetic Python boundary-validation task: accept integers from zero through ten and reject values outside that range. It makes the model's proposed change and its test evidence inspectable.
How it works
The prototype sends the fixed task to NVIDIA Nemotron 3 Ultra through Nebius Token Factory. It constrains the model response to a plan, a fixture path, and a compact function. A restricted abstract syntax tree interpreter checks allowed integer comparisons and Boolean expressions against six boundary cases. It records the proposal, patch, test results, response hash, and provider usage in a receipt.
What we observed
In a live run on October 30, 2026, the application received HTTP 200 from Nebius and verified the response model ID against nvidia/Nemotron-3-Ultra-550b-a55b. The first model proposal passed all six boundary cases. This is one limited result, not a broad reliability evaluation. The live run did not demonstrate a failure followed by a repair or Docker isolation.
A separate deterministic offline fixture illustrates the loop mechanics with scripted proposals. It is not live model evidence.
Limitations and access
The application uses Python's standard library. Its restricted interpreter is not a general-purpose operating-system sandbox. Credentials are supplied through the environment; the observed private run did not retain the API key or raw provider response in its receipt. This is a prototype, not a production security tool.
Public code, a judge-accessible demo or test build, and a video are being prepared and are not yet available.
Log in or sign up for Devpost to join the conversation.