VeriLab

A software layer that checks lab robot code before it runs, catching mistakes that would slip past standard safety checks and ruin a real experiment.

Comment

Inspiration

I've always wanted to build at the intersection of AI and biology, and I've done research in actual bio labs, so I've watched this exact failure happen firsthand. A professor at Stanford ran AI-generated robot code that broke experimental equipment, and that stuck with me. Labs are starting to let LLMs generate the Python that drives their liquid-handling robots, and the verification story around that is basically nonexistent.

The industry-standard tool, the Opentrons simulator, only checks that generated code is syntactically valid and executes without throwing, which is a runtime correctness guarantee, not a semantic one. I proved this to myself on day one. I wrote a protocol transferring 500 microliters into a 360 microliter well, ran it through the simulator, and got a clean exit code. No exception, no warning, nothing. The simulator has zero concept of physical invariants like container capacity.

That's why I built VeriLab. It closes a real gap in an industry that quietly shapes everything in our lives, from food to medicine.

What it does

VeriLab is a precondition-verification layer inserted between an LLM's output and physical robot execution. A scientist provides a protocol in unstructured English. An extraction pass, the only stochastic component in the whole pipeline, parses that into a structured intermediate representation: typed steps carrying reagent, volume, source, destination, and temporal metadata. Everything downstream of that IR is pure, deterministic, side-effect-free validation. A capacity checker walks the step sequence and tracks cumulative volume state per well against ground-truth labware capacities resolved from the real Opentrons catalog, not inferred or hallucinated. A separate pass enforces a causal ordering constraint, no read or aspirate operation may occur before its corresponding write, i.e. no well can be drawn from before something has actually been dispensed into it. A third check operates at a different layer entirely: it instruments the generated code's aspirate and drop_tip calls via monkey-patching and executes it directly, tracking tip-to-reagent state transitions to catch cross-contamination that's invisible in the static IR. Only an artifact that passes every gate gets compiled into real Opentrons Python, which then goes through the actual simulator, and any simulator exception gets fed back into a bounded self-correction loop, up to three regeneration attempts, before escalating to a human. A human always signs off before anything touches physical hardware.

How I built it

The backend is FastAPI calling the real Anthropic API, and every generated protocol is validated against the actual Opentrons simulator, not a mock. Claude Sonnet 4.6 only shows up in two spots: turning English into structured steps, and repairing generated code against a real simulator error. Everything else is deterministic Python with one correct answer, capacity lookups against the real Opentrons catalog, volume tracking, step ordering, tip tracing, deck assignment, and pipette selection.

Challenges I ran into

The worst bug class by far was referential drift in the extractor's output: the model would declare a labware object under one identifier and then reference it under a different, more generic one somewhere in the step list, silently breaking every downstream dictionary lookup keyed on that ID. I went through eight distinct mitigation attempts, tightened prompting, a validation pass that just logged the mismatch, type-constrained fuzzy matching, a full model swap, before landing on the fix that actually generalized: stop trying to enforce self-consistency at generation time, and instead treat the subset of correctly-referenced steps as a ground-truth signal to disambiguate the broken ones post hoc, essentially a lightweight coreference resolution pass running entirely outside the model. The second hard problem came from a real protocol with repeated bind-wash-elute cycles reusing the same physical wells. My IR had no representation for volume leaving a well, only entering one, so the capacity checker treated every wash step as a pure addition with nothing ever decremented, and reported a fabricated 3500-plus microliter overflow. Adding a first-class discard operation to the schema, and making the checker treat a null volume as a full evacuation of that well's tracked state, fixed an entire class of protocols in one pass.

Accomplishments that I'm proud of

I benchmarked this against a zero-shot single-prompt baseline on identical inputs. Zero of four naive completions simulated successfully. Four of four passed through this pipeline. I validated against nine real published protocols beyond my original dev set, and I'm reporting two of them as unresolved hard cases rather than quietly excluding them from the numbers, because a validation set you've curated to only show wins isn't a validation set. And I got the self-repair loop to fire on a genuine, unseen simulator exception, not a unit test fixture, watched it diagnose a missing trash bin declaration and patch exactly that one line, nothing else touched.

What I learned

The biggest lesson was architectural: the moment you notice yourself hoping a language model will just remember something correctly, that is the signal to stop hoping and push that decision into code instead. I also learned that a real evidence base, actual numbers from actual tests, actual documented failures, is worth far more to a technical reader than a demo that only shows the happy path.

What's next

Opentrons already ships their own AI protocol generator, so I am not trying to out-build a company that owns the hardware. The path forward is positioning VeriLab as the verification layer underneath tools like that, expanding it to check protocols against other robot platforms, and eventually proposing it as an open standard other labs and manufacturers could adopt rather than something only I maintain.

Built With

  • anthropic-api-(claude-sonnet-4.6)
  • css
  • fastapi
  • html
  • javascript
  • json-schema
  • opentrons-python-api
  • opentrons-simulator
  • python
  • subprocess-instrumentation
Share this project:

Updates

Submission history