Integration Verifier

Find the fallback behind a claimed live AI feature.

Comment

The problem

An app can display a convincing AI answer while silently returning a mock or fallback. Integration Verifier tests the connection between the provider response and what the browser actually shows.

The product

Run a bundled Node application under unavailable, malformed-response and unique-marker provider conditions. Inspect browser evidence, identify a hidden fallback, then apply the bundled deterministic repair and replay the unchanged regression in a temporary copy. Original source stays intact. This version deliberately targets a controlled sample; it does not claim arbitrary-repository repair.

Sponsor technology and build

The application uses Node.js, JavaScript and Playwright. A real Tavily search retrieved vendor documentation, and NVIDIA Nemotron Lightning (nvidia/Nemotron-3_5-Lightning) on Nebius Token Factory produced a checklist from selected passages. Search and inference have separate receipts; external text is advisory input, never executable instructions.

A live generated-repair attempt ran on 30 October 2026: NVIDIA Nemotron Lightning on Token Factory generated the candidate, then Token Factory Sandboxes ran the fixed regression. The original sample exposed its hidden fallback (two failures and one successful marker case). The candidate contained a JavaScript syntax error, so all three post-repair cases failed before app readiness. The remote operation completed with process exit 1 and the application kept the repair failed/unverified. No successful generated repair or complete post-repair browser outcome is claimed.

Deepgram Aura-2 Thalia generated the narration audio. Decoding, non-silence and clipping checks passed. The public edited demo uses Deepgram narration and visibly discloses the rejected candidate; listening review is not claimed.

What is verified

The actual local browser journey detected a hidden fallback. Its deterministic repair passed all nine evidence checks: the unchanged regression failed before the patch and passed afterward. The standalone full suite passed 152 tests; later focused checks passed 17 research units, two API/browser checks and four SDK checks. These checks overlap and use local fixtures, separate from the live research receipts.

The research contains a supported refusal-handling proposal, but its broader summary exceeds its selected quote. Original outputs remain preserved; a separate assistant review proposes narrower checks without claiming they were executed. Literal citations and successful API transport do not establish semantic correctness.

Run and current access

Public repository · Exact-commit test-build ZIP

Published and checked on 30 October 2026 at commit 0022b49d5eba87539bbcf97e479913781023a172. The public repository includes an MIT license, setup documentation, source and tests. The exact-commit ZIP was prepared from this release; the public archive URL was checked without authentication.

Download the ZIP, unzip it, enter the extracted folder and run the commands below. Node.js 22+ is required. Local controlled examples work without provider keys. Live sponsor actions require private user-owned environment configuration, verified provider access and a fresh free-usage review; no credentials are bundled.

npm ci
npx playwright install chromium
npm start

Open http://127.0.0.1:4190/verifier. Run npm test without provider credentials for local validation. The controlled browser and deterministic repair workflow run locally. A successful live generated repair in Token Factory Sandboxes remains unverified.

This download is a local source test build. The server binds to loopback; no always-on hosted application or organizer-managed provider credentials are supplied. The public demo video is attached below. It distinguishes the rejected live generated candidate from the separately dated successful controlled local replay.

AI tools and development

Codex assisted with implementation, interface refinement, test authoring, debugging and independent review. NVIDIA Nemotron performed the bounded runtime tasks described above; Tavily supplied search results, and Deepgram generated narration. AI output was checked against saved state, validation rules and actual execution evidence. Separate Claude-built hackathon projects are not part of this entry. No controlled head-to-head comparison with other models was performed.

Public demonstration

Watch the English demo. It uses edited actual browser captures with dated evidence, Deepgram narration, and explicit limits. Full technical decode and selected visual-frame checks passed.

Built With

Share this project:

Updates

Submission history