ReproFrame AI

Scientific visuals you can verify, not just admire.

Comment

Inspiration

Scientific diagrams are visually persuasive but difficult to audit. A polished generated figure can silently add unsupported claims, lose source context, or become impossible to reproduce after one prompt revision. ReproFrame AI treats every visual as a verifiable research artifact instead of a disposable image.

What it does

A researcher supplies bounded claims, exact labels, prohibited elements, an audience, and a visual direction. ReproFrame AI then:

  1. generates a self-contained scientific SVG with Google Gemini;
  2. runs mandatory deterministic checks for every exact label, evidence-backed claim, prohibited element, and integrity field;
  3. passes explicit feedback into a Genblaze AgentLoop when a candidate fails;
  4. sanitizes SVG against scripts, event handlers, external URLs, foreignObject, and oversized payloads;
  5. optionally applies a model-based visual rubric for legibility, hierarchy, overlap, and claim separation;
  6. stores the brief, every candidate, evaluations, Genblaze manifests, SHA-256 hashes, and final manifest in private encrypted Backblaze B2;
  7. exposes a durable run ledger, attempt comparison, byte-level verifier, protected human decision, and downloadable proof bundle.

The result is not merely an image. It is an inspectable chain from evidence constraints to accepted bytes.

How we built it

The application is a Python/FastAPI service packaged in a digest-pinned Docker image and run as non-root UID 1000. Genblaze orchestrates generate → evaluate → feedback → retry and preserves canonical manifests for each attempt. Google Gemini gemini-3.1-flash-lite generates safe SVG candidates and performs the optional visual review. An authenticated GMI/Seedream adapter is also implemented, but the verified public path uses Gemini because the GMI account currently has no generation credits.

Backblaze B2 is the durable system of record, not a final-output attachment. Every attempt writes its candidate bytes, evaluation record, and Genblaze manifest before the final ReproFrame manifest is committed. Private objects are served through a narrow run-scoped proxy; credentials and signed storage URLs never enter the browser or manifests.

A deterministic fixture provider keeps development, CI, and the five-case benchmark fully offline. Provider and model metadata is explicit, so fixture evidence is never presented as a live-model result.

Verified public run

The hosted app is available at reproframe-ai.onrender.com.

Run 7e65b640-a45d-4c91-bc47-276b4fdc27a5 visibly demonstrates the retry loop:

  • attempt 1 scored 83% and failed because two required claims were not exact;
  • Genblaze carried that feedback into attempt 2;
  • attempt 2 scored 100% and passed all Gemini visual-review checks;
  • the verifier re-fetched and validated the canonical manifest, two asset hashes, and two Genblaze manifests across nine B2 objects;
  • the downloadable proof bundle contains ten files and passes ZIP integrity testing.

This is a narrow reproducibility demonstration, not a claim that one run measures general model quality or scientific truth. The repository also includes five deterministic communication fixtures with clearly scoped metrics.

Challenges

The hardest part was preserving honest provenance across multiple providers and retries. A retry must not overwrite evidence from an earlier failed attempt. We also had to make generated SVG safe to render, distinguish manifest validity from byte-level asset verification, keep B2 private, and provide a useful public demo without paid GMI credits.

Accomplishments

  • real Genblaze AgentLoop control, not a token SDK import;
  • B2 as durable orchestration and evidence storage;
  • deterministic gates plus an optional model-based visual rubric;
  • exact stored-byte verification and portable evidence bundles;
  • visible attempt comparison and explicit human acceptance;
  • 12 passing tests, a successful secret scan, a locked container smoke test, and green GitHub Actions CI;
  • a live, judge-accessible Docker deployment with protected server-side credentials.

What we learned

Generative-media quality is not only a model problem. Clear evidence constraints, deterministic validation, immutable artifacts, byte-level verification, and honest provider metadata make outputs substantially more useful for research and technical communication.

What's next

Next steps are version-to-version visual diffs, richer scientific-domain rubrics, more GMI media workflows when credits are available, and controlled user studies comparing evidence-bound generation with ordinary prompt-only workflows.

Source code and evidence

Built With

Share this project:

Updates