API Repair Bench

Untrusted code, bounded repair: NVIDIA Nemotron proposes via Nebius Token Factory; Nebius Sandbox runs declared checks; one correction maximum, then human review.

Comment

Try the project

Live repair workspace · Recorded evidence replay · 1:29 uninterrupted demonstration · Public source

Who it is for and why it matters

For integration engineers fixing API clients that return HTTP 200 but silently lose or corrupt data. A successful HTTP response is not a successful contract. The supplied fictional incident is a client that returned page one and silently omitted page two. Before: missing records with no failure. After: every page is returned in order, duplicate occurrences are preserved, cursor cycles are rejected and a ten-request boundary stops an unbounded fetch. These are declared checks, not customer-outcome claims.

Try one workflow in two minutes

  1. Open the live workspace. Pagination repair is selected.
  2. Click Run live repair once and follow NVIDIA Nemotron → Nebius Sandbox in the trace.
  3. Inspect the proposed function, named checks and final stopping reason. Download the evidence for review. No patch is deployed.

The access window includes January 1, 2027, Pacific time, within the available project credit. Admission permits two jobs per browser in a rolling 24 hours and one active job globally. Browsing makes no model call. If live capacity is unavailable, the recorded demonstration and archived correction remain accessible.

Watch the project work

The polished October 23 1:29 demonstration shows two separate, uninterrupted October 13 browser runs at their recorded speed. The opening identifies the broken contracts before showing the live workspace; narration and callouts explain the consequences while the actual browser trace remains visible.

  • Identifier repair: the browser submits a function that turns the account code "001" into 1. NVIDIA Nemotron proposes a replacement through Nebius Token Factory. Nebius Sandbox returns 3/3 declared checks passing on the first proposal. The desk shows the generated function, named checks, stop reason, and evidence download.
  • Pagination repair: a client can return HTTP 200 while silently dropping page two. A fresh browser-triggered run reaches 4/4 declared checks on the first proposal, with the trace and downloadable evidence visible in the same workspace.

The video does not show a correction; neither new run needed one. A separate saved replay and technical evidence document the September 11 invoice attempt that went from 4/9 to 9/9 after one bounded correction. These are fictional inputs and observed runs, not a reliability estimate or customer claim. The model cannot deploy a patch or grant itself extra attempts.

What it does

API Repair Bench turns a narrow Python repair into an inspectable sequence: propose → remote Sandbox checks → at most one correction after a completed failed proposal → stop for human review. The desk shows the generated function, named check outcomes, progress trace and downloadable evidence together. Nothing is deployed automatically.

The /live/ workspace runs a fresh browser-triggered workflow using NVIDIA Nemotron through Nebius Token Factory and Nebius Sandbox. Choose a supplied example or paste one Python function with JSON contract checks. Browsing makes no model call; clicking Run live repair dispatches a job within the operator's remaining allowance. Live availability is bounded, and the homepage remains a separate, no-cost recorded replay when the allowance is unavailable. The site requires no sign-in.

The 60-second structural pitch

HTTP 200 can still drop page two or corrupt an amount. A plausible AI patch is not proof. API Repair Bench separates proposal from judgment. NVIDIA Nemotron, via Nebius Token Factory, proposes one replacement function. The controller checks proposal format, binds contract and proposal with SHA-256 and a run ID, and runs candidate code in a remote Nebius Sandbox, away from the website Worker. For built-in cases, expected outcomes stay outside the uploaded bundle. The controller compares returned observations, names failed checks, and allows one correction against the same contract. Truncated model responses and uncertain provider effects stop rather than being silently retried. A pass creates evidence for human review; nothing deploys. The trust boundary is explicit: candidate and observation runner share a Sandbox process, so results are not independently attested execution.

Three systems-engineering questions judges should ask

1. Can a malicious candidate forge a passing result? Yes, in principle. Remote execution separates the candidate from the website Worker, and built-in expected results stay outside the uploaded bundle. But candidate and observation runner share a Python process. Returned stdout is therefore self-reported, and 9/9 means the reported observations match the contract, not that execution was independently attested or the function is generally correct. Receipts explicitly record trusted_execution_proven: false. Stronger isolation between candidate and observer, with authenticated observations, is the next execution boundary.

2. What do the SHA-256 bindings actually prove? Canonical challenge and proposal digests, per-file digests, upload-byte checks, and run identifiers bind the declared artifacts at the boundaries the controller checks. A correction must reference the same challenge. These checks expose mismatches or substitution within that workflow. They are not signatures, remote attestation, or proof that self-reported observations are honest.

3. What prevents duplicate effects, endless corrections, and runaway spend? An incomplete model response cannot proceed to Sandbox. A completed failed baseline may authorize one correction against the same contract; a further failure stops. Durable reservations, a single-active-job constraint, phase tokens, and leases constrain progression. Unknown outcomes of effectful provider calls become uncertain and are not automatically resent. The app never deploys a patch. Its allowance controls admission and reserved spend; it is not a provider-enforced, account-wide billing cap.

Inspiration

Review desk for a broken API client function when HTTP succeeded and the contract did not. A conversion can erase leading zeros, a pagination helper can drop records, and a retry can repeat an expired token. A plausible patch is not proof that the integration is fixed.

How we built it

The JavaScript service defines contracts, validates proposals, reserves runs durably and compares returned observations. A Python observation runner executes with the candidate inside Nebius Sandbox. HTML, CSS and JavaScript provide the review desk; provider credentials remain server-side.

The model is NVIDIA nvidia/Nemotron-3_5-Lightning, accessed through Nebius Token Factory. Managed inference removed the need to provision a GPU. Nebius Sandbox executes the candidate remotely, keeping generated code off the development Mac. For built-in cases, expected outputs remain outside the candidate runtime.

Model calls use explicit response allowances, strict proposal JSON and no automatic retries on uncertain attempts. A challenge hash covers source and contract; a proposal hash binds the candidate to its evaluation. Runtime output is bounded and retained. Actual exception diagnostics can inform one correction without silently changing the contract.

Challenges and integrity

The evidence retains rejected and incomplete model responses, interface mismatches and failed checks alongside successes. A truncated response is not admitted as runnable code; a failed correction stops. This makes uncertainty inspectable instead of hiding it behind a green demo.

Limits and next steps

This is a small synthetic evaluation and a single-function review tool. Candidate code shares a process with the observation runner, so matching returned observations are not tamper-proof execution attestation. Broader integration cases and stronger separation between execution and trusted measurement are next.

Built with AI assistance under founder review. The project contains original hackathon code and fictional fixtures, not private DeltaX engine code or customer data. The video uses a built-in synthetic narrator; no personal voice was cloned.

Usability validation

A concise integration-engineer test protocol and feedback path are published. The test asks reviewers to identify the original data loss, explain the stopping reason and download the bound evidence without coaching. A structured external usability study has not yet been completed; no participant findings or customer adoption are claimed.

Built With

  • javascript
  • nebius
  • nebius-sandbox
  • nebius-token-factory
  • nvidia-nemotron-3-5-lightning
  • python
Share this project:

Updates

Submission history