AutoML Autopilot

An ML repair agent that refuses to trust green tests: Nemotron patches the repo in a sandbox, then an independent verifier checks train/serve parity, leakage and holdout quality.

Comment

Inspiration

Most ML production failures don't crash. A feature is reordered in the serving path, a refactor quietly uses a column that only exists after the outcome, or a library upgrade changes preprocessing. The unit tests stay green, the model keeps answering, and its predictions are wrong.

Coding agents make this worse. Their stop condition is "the tests pass", so they will happily weaken a test or patch the symptom. We wanted an agent whose work is judged by something it cannot talk its way past.

What it does

You give AutoML Autopilot a Python ML repository (GitHub URL, zip or a bundled sample) and a goal ("online scoring raises after the feature refactor"). It:

  1. Freezes a contract. autopilot.yaml declares commands, data, editable paths and ML invariants. It is hashed into an Engineering Change Contract, and the agent cannot change it.
  2. Reproduces the failure in an isolated Docker sandbox (no network, non-root, resource limits).
  3. Diagnoses. Nemotron proposes competing root-cause hypotheses. Each one carries a falsifiable probe that the platform checks for free on the untouched code, and hypotheses are re-ranked before any coding tokens are spent. Tavily is consulted only when the traceback looks like a library API change.
  4. Repairs in parallel. Each hypothesis gets its own sandboxed candidate branch. The agent must write a new regression test first. Once that test has failed, the agent can no longer edit it, so it has to fix the code instead.
  5. Verifies independently. A separate verifier service, which the agent cannot reach, grades the smallest eligible patch:
    • Code oracle: diff policy, a clean apply onto the pristine commit, fail-to-pass and pass-to-pass against a baseline it measures itself.
    • ML oracle ("predictions out, labels in"): the candidate's own train/predict pipeline runs on holdout features only. The verifier then checks train/serve parity, availability at prediction time (it blanks post-outcome columns and re-scores), and per-group guardrails. Only then does it open a single-use holdout for the paired-bootstrap non-inferiority test.
  6. Delivers a signed receipt, a .patch and an optional draft pull request. Nothing merges automatically.

How we built it

  • Backend: FastAPI, async SQLAlchemy, Postgres, Alembic, pydantic v2.
  • Seven services: web, api, repair-worker, verifier, sandbox-runner, Postgres, and the v4 investigator worker. They sit on segmented networks.
  • Least privilege:
    • Only the repair-worker holds model keys.
    • Only the verifier holds labels and the holdout ledger.
    • Only the sandbox-runner touches Docker, and it holds neither keys nor labels.
  • Frontend: Next.js, React, TypeScript and Tailwind, with Playwright and axe accessibility tests. Recorded runs replay at zero inference cost.
  • Fixtures: six faulty ML repositories (feature-order skew, target leakage, segment regression, serialization, sklearn API change, prompt injection). Each comes with a three-way proof: the unpatched code fails, a plausible wrong fix passes the tests but is rejected, and the reference fix is verified.
  • Product layer: GitHub sign-in with workspaces and roles, a GitHub App webhook ("CI failed → open a repair task"), spend caps with atomic reservations, metrics and retention.

The paired non-inferiority decision accepts a candidate $c$ against the base $b$ when the lower 95% bootstrap bound of the improvement clears the margin:

$$\Delta = m(c) - m(b), \qquad \text{accept if } \; Q_{0.025}\big(\Delta^{*}\big) > -\varepsilon$$

For error metrics (RMSE, MAE) we use the relative improvement $(m(b)-m(c))/m(b)$, so $\varepsilon$ stays unit-free.

Challenges

  • Green tests lie. In our first live run, both candidates made every test pass. Both were wrong fixes, and the ML oracle rejected both. That moment became the thesis of the product.
  • Agents game their own tests. Nemotron once loosened the parity test it had just written. We added the own-test lock and a "test first" rule.
  • Infrastructure honesty. An early version reported a sandbox crash as REJECTED, which blamed a correct patch. Infrastructure failures are now retried once and then reported as INCONCLUSIVE, never as a rejection.
  • Isolation across containers. We hit Docker-out-of-Docker path mapping, file capabilities on candidate-written files, and grader probes running next to candidate code. The parity probe now only records raw outputs, and the verifier makes the comparison.
  • Provider availability. Long evaluation runs hit timeouts and HTTP 503s, so slots end INCONCLUSIVE rather than counting as successes.

What we learned

  • The verifier matters more than the agent. A strong model that grades its own work is still a liability.
  • Most useful "AI safety" here is ordinary engineering: frozen contracts, labels the agent can never read, a holdout that can be opened only once, and receipts a reviewer can check.
  • Report the denominator. An unsuccessful repair reported honestly is worth more to a reviewer than a confident wrong one.

Results so far (genuine runs, NVIDIA-hosted Nemotron)

  • Full-agent pilot on six fixtures: 3 VERIFIED (churn-service, loan-scorer, refund-risk) and 3 INCONCLUSIVE after provider failures. There were 0 false accepts.
  • The churn-service repair went end to end through the full 7-service stack. The verified fix replaced a hand-copied serving column list with the shared feature order. Holdout ROC AUC was 0.9114, equal to the pre-bug baseline. It took about 10 minutes and about $0.14 at Nebius-equivalent rates.
  • Wrong fixes that passed every test were rejected by the ML oracle.

Built With

Share this project:

Updates

Submission history