RTLRefine

Repair RTL with NVIDIA Nemotron on Nebius, independently check patches, and export runnable regression tests to catch returning bugs.

Comment

Inspiration

An AI-generated RTL patch creates a difficult question: did it fix the behavior, or did it just satisfy the test that exposed the bug?

As an RTL designer, I wanted to inspect the proposed change, reproduce its results, and keep a regression test that would catch the same defect later.

A UART example made this problem concrete. Its transmitter and receiver could pass a loopback test while exchanging bytes in the wrong order for an external peer. The test passed because both sides agreed with each other.

I built RTLRefine around that gap between a passing demonstration and evidence an engineer can review.

What it does

RTLRefine uses NVIDIA Nemotron through Nebius Token Factory to propose Verilog/SystemVerilog repairs.

Users upload a ZIP containing their RTL and self-checking testbenches, configure file roles in the browser, and reproduce a public failure. The model can edit permitted design files and revise once using public execution feedback. Testbenches, interfaces, and configuration remain protected.

Additional audit benches or profiles stay outside model context and run after patch selection is fixed. A checked download becomes available only after the required checks and the exact exported checker pass.

The exported project includes:

  • The selected RTL and source differences.
  • Verification logs, model responses, and file hashes.
  • A standalone regression runner with JSON and JUnit reports.
  • A GitHub Actions workflow.

Engineers can rerun these regressions after changing the RTL without installing the application or supplying a model key.

For a supported synchronous FIFO contract, RTLRefine also finds bounded symbolic counterexamples beyond smoke tests, reproduces them independently, and exports them as portable regressions.

How I built it

I built the browser workspace with Python and Streamlit. Icarus Verilog, Verilator, and Yosys support the native checking workflows.

The general project importer handles multiple source files, dependencies, functions, parameters, wide signals, and testbench-driven clocks within the supported simulator subset.

Native verification workers run with read-only inputs, resource limits, and no network access or API credentials. Fresh compilation and execution connect acceptance to the actual exported source.

The FIFO workflow uses a behavioral monitor and replays counterexamples in Icarus against an independent Python queue. It does not require corrected RTL as a reference implementation.

The standalone checker bundles the verification engine and its license. Hashes bind its checks and configuration, while repository review governs intentional changes to those files.

Challenges I ran into

Plausible explanations did not reliably produce correct patches.

The UART development required six actual model requests across three runs. Attempts included an incorrect diagnosis, a compilation failure, and a truncated reasoning response. I retained those records, improved the ordering of public feedback, and allowed a bounded reasoning revision with a larger completion limit. The final two-request run passed the unchanged checks.

Uploaded projects introduced different problems: identifying testbenches, preserving compile order, recognizing meaningful completion, and clearing old success results when a new check failed.

I exercised the full browser workflow through automated upload, inspection, download, reimport, and regression runs, then fixed the issues those checks exposed.

Accomplishments I'm proud of

  • Reproduced a known historical UART bit-order issue using originally unmodified, commit-pinned external RTL. Its limited loopback passes, but an independent LSB-first peer receives 0x80 instead of 0x01. The selected NVIDIA repair passes eight profiles with 186,772 assertion evaluations.
  • Repaired a separate known verilog-axis throughput regression with one actual NVIDIA request. The result passed eight testbenches with 30,493 evaluations.
  • Demonstrated exported regressions on four existing projects: repaired source passes, reintroducing the defect fails, and restoring the repair passes again. GitHub Actions runs the exported commands too.
  • Found and independently replayed a five-state failure in an authored FIFO example that passed its smoke tests. Its repair retains portable regressions and bounded final checks.
  • Reduced complete check-and-export time by 11.8–19.4% across four existing inputs, using four paired measurements per input on one host. Decisions, assertion counts, and exported project contents matched. These timings exclude model inference.

What I learned

Acceptance depends on the behavioral contract and the quality of its checks. Loopback can hide a shared mistake; an independent peer can expose it.

A useful repair also needs evidence that survives the application session. Exporting the checks lets another engineer inspect the result and detect the defect if it returns.

These demonstrations are selected development cases, not an unseen repair-accuracy benchmark or new bug discoveries. The UART and verilog-axis cases share an upstream author. The browser audit was automated rather than a study with independent designers.

General project results cover supplied finite tests. FIFO results are bounded and contract-specific. They do not establish unbounded correctness, physical timing, power, or CDC safety.

What's next for RTLRefine

I plan to evaluate the complete workflow with independent RTL designers using their own projects, expand the supported behavioral contracts, and improve support for larger designs.

I also want to compare repair strategies using predeclared cases, matched budgets, and total workflow cost, with failed attempts included in the results.

Built With

  • bubblewrap
  • c++
  • doker
  • icarusverilog
  • nebiustokenfactory
  • nemotron
  • nvidia
  • python
  • streamlit
  • systemverilog
  • verilator
  • verilog
  • yosys
Share this project:

Updates

Submission history