Inspiration
Release Witness is a Best Apps and Agents entry for small product teams without dedicated QA. NVIDIA Nemotron 3.5 Lightning runs through Nebius Token Factory to map each change to the complete reviewed check catalog; isolated Chrome assertions still own every verdict.
Small teams often review a pull request with a mix of intuition, stale checklists and screenshots that are difficult to reproduce. A passing test suite can still miss the workflow that matters, while an AI summary can sound confident without having observed the product. Release Witness was built to join those pieces: execute a bounded browser contract, preserve the evidence and let a model advise only within what was actually checked.
What it does
Release Witness turns a developer's change description into a bounded risk map, then runs a baseline and candidate build as one release-review action using reviewed JSON workflow manifests. Each check runs in a fresh Chrome context and records the risk hypothesis, reproduction steps, expected and observed behavior, screenshots and console errors. The resulting report distinguishes pass, fail and not-tested coverage. It opens a comparison that classifies each check as resolved, regressed, unchanged or unverified, places changed evidence side by side, and freshly verifies both SHA-256 receipts against the reports and every screenshot on disk.
The included appointment demo starts with two seeded defects: a booking disappears after reload and a sold-out time can still be selected. The candidate repairs both while preserving required-name validation. A smaller Fieldnotes workflow demonstrates that the same model contract and runner execute another reviewed application without scenario-specific branches. Each comparison now ends in a deterministic, receipt-gated posture: supported within tested scope, hold within tested scope, or review required. The rule cannot approve missing evidence and always repeats how much coverage remains unverified.
The first users are small product teams reviewing pull requests without a dedicated QA function. The product gives the reviewer a compact answer to three questions: what risk did the change create, what did Chrome actually observe, and which claims remain untested? A one-click judge tour opens the strongest saved Nemotron-backed comparison without requiring credentials or spending credits.
Judge quick path: open the credential-free demo, choose Start 90-second tour and follow its six steps through the recorded model risk map, Chrome observations, bounded advice, verified delta, receipt-gated release posture and portfolio receipt. The posture step waits for comparison evidence before focusing the bounded decision and explicit unverified boundary. The final step announces its new-tab transition before opening the certified two-workflow aggregate with returned-model verification and the exact release source. The first-screen source, video, receipt and readiness resources also announce their new-tab transitions. The hosted tour replays saved real Nebius calls; it makes no fresh inference request.
Why the idea is different
The model's authority stops before the verdict. Nemotron must map the change to every reviewed check, but the manifest defines coverage, Chrome produces observations and deterministic assertions decide outcomes. The product's final artifact is therefore a release receipt that connects model reasoning, browser evidence, explicit unknowns and exact source provenance. A fluent model answer cannot turn missing evidence into confidence.
How we built it
The application uses a loopback Node.js server, a dependency-light browser UI, Playwright with Google Chrome and validated JSON manifests. An allow-listed interpreter supports navigation, form actions, screenshots and explicit assertions. Reports are written atomically and survive restart.
NVIDIA Nemotron 3.5 Lightning runs through the Nebius Token Factory API. The first bounded call maps the change to a specific risk hypothesis for every known check and orders the complete catalog. After execution, the second call chooses an advisory next action from server-provided options. The model cannot add or omit a check, rewrite a browser result or turn unknown coverage into a pass. Invalid or unavailable model output is shown explicitly while the browser evidence remains usable.
Challenges
The hardest part was keeping model assistance useful without letting it become the source of truth. Early free-form advice could invent unsupported work. We replaced it with strict schemas, allow-listed identifiers and server-owned wording. We also had to preserve useful reports across provider failures, browser failures and restarts, and distinguish an observed failure from a check that never ran.
Accomplishments
- Detected 3/3 declared seeded defects and classified 3/3 repairs across two controlled workflows, while preserving three passing invariants, reporting zero observed regressions and keeping both excluded-coverage boundaries unverified.
- Verified the required NVIDIA/Nebius runtime across both applications: four receipted Nemotron 3.5 Lightning runs through Nebius Token Factory, 12 grounded risk hypotheses, four allow-listed advisories, 2,848 tokens and 7.49 seconds of measured model work.
- Demonstrated reuse on Harbour Appointments and Fieldnotes through validated manifests; the same model contract, browser runner and receipt path execute both without scenario-specific runner branches.
- Added a server-derived portfolio verifier that checks declared ground truth, durable pair relationships, four versioned run receipts, all 16 screenshot files and four receipt-bound Google Chrome execution records before certifying the public 3/3 result. Its top-line receipt reports supported workflows beside verified workflows and derives one bounded portfolio posture with the exact unverified-boundary count; its workflow cards show their receipt-gated posture and exact unverified boundary, both explicitly covered by the aggregate SHA-256, and directly download pair receipts bound to the running version, immutable source commit, source-archive hash, evidence-archive hash, live readiness decision and current portfolio digest; other saved comparisons are explicitly not portfolio-certified.
- Bound the homepage's judge-facing outcome totals to a visible certification row with the exact current portfolio SHA-256 and a direct receipt link; incomplete evidence shows
PORTFOLIO NOT CERTIFIEDin the same place. - Kept incomplete model evidence honest: a verified subset is labeled partial with its exact workflow count, while a no-model state explicitly claims no model result. Neither state can enable the certified judge tour.
- Extended the same fail-closed language to the printable benchmark receipt, which changes both its portfolio heading and Nemotron summary when evidence is incomplete.
- Bound the exact release, immutable Git commit and both downloadable archive SHA-256 digests into a responsive, printable benchmark receipt with direct links to both comparisons and all four source run-receipt digests. Its four external provenance links consistently open in new tabs and announce that transition for assistive technology.
- Published a fail-closed readiness receipt that returns 200 only when live release provenance, every supported workflow posture and the complete workflow, screenshot, model and Chrome evidence chains verify; otherwise it returns 503 with specific failures. Its successful summary exposes supported-workflow and unverified-boundary totals.
- Designed a one-click judge path from bounded model risk to browser observations, side-by-side repair evidence, explicit unknowns and the source-bound portfolio receipt, verified on desktop and at 390 × 844 without horizontal overflow.
- Added a deterministic, receipt-gated release posture to the comparison and downloadable receipt: support requires two verified evidence receipts with no regression or remaining failure; incomplete evidence requires review, and every state preserves the unverified boundary count.
- Covered model schemas, provider failures, tampering, unsafe targets, concurrency, restart and deployment recovery, paired execution, receipt-gated release posture, public-demo isolation and the real-Chrome judge journey with 26 passing tests. Public CI also parses the deployment script, audits production dependencies and runs both workflow smokes in Chrome.
What we learned
AI is most credible in release review when its authority is narrower than the evidence system around it. Structured choices, deterministic execution and visible unknowns produced a more useful result than asking a model to judge an application from prose alone.
What's next
Next steps are authenticated team workspaces, encrypted evidence storage, more browser engines, reusable manifest tooling and integrations that attach a report to a pull request. Public hosting also needs authentication, tenant isolation, retention controls and a safe worker architecture before it can accept arbitrary projects.
Built with
Node.js, JavaScript, Playwright, Google Chrome, NVIDIA Nemotron 3.5 Lightning and Nebius Token Factory.
Nebius and NVIDIA feedback
Token Factory's OpenAI-compatible chat-completions interface made initial integration straightforward, and serving Nemotron without managing GPU infrastructure kept the prototype focused on the product. Model identity and token-usage fields were useful for preserving evidence. Nemotron produced useful prioritization when the task was tightly constrained, but concise structured output required disabling extended thinking and validating every identifier. Clearer model-specific examples for structured responses, reasoning-token behavior and trial-versus-billing status would reduce integration time. The product therefore treats provider output as bounded advice and keeps browser evidence useful when an inference fails.
Submission-period work
Release Witness was created during the hackathon submission period. The application, manifests, runner, model contracts, evidence format, CLI and publication package in this repository are the competition work.
Built With
- chrome
- javascript
- nebius
- nebius-token-factory
- node.js
- nvidia-nemotron
- playwright