FIELDREEL

Most AI tools stop when media is generated. FIELDREEL starts there: it turns construction-inspection footage into verified, traceable evidence — generation proposes, a human decides.

Comment

Inspiration

Generating media is no longer the hard part. In an inspection workflow the hard questions are the ones a generated image cannot answer by itself:

Where did this come from? What source supports it? Did the model preserve what was actually observed, or invent something? Who reviewed it? Can the final report be traced back to the original footage?

A construction inspector captures dozens of photos and videos on a site visit. The bottleneck is not a lack of media — it is that nothing connects source media → observation → generated media → verification → final artifact.

FIELDREEL is that missing layer.

What it does

Turns raw site media into a verified, traceable evidence package.

RAW MEDIA → UNDERSTAND → GENERATE → EVALUATE → VERIFY → PRESERVE
  1. Upload — video and photos go straight from the browser to Backblaze B2. The API never touches the bytes.
  2. Ingest — FFmpeg extracts frames and a streamable proxy; every derivative is checksummed and stored in B2 under a key that names its source.
  3. Understand — GPT-4o, orchestrated through Genblaze, reads the frames and proposes candidate findings. Never facts. Language is hedged automatically: "Possible cracking on the north façade".
  4. Evidence — for each candidate, FIELDREEL selects supporting frames, generates crops and annotated visuals, and scores them.
  5. Evaluate — an evidence engine measures source consistency, relevance, visual quality, context sufficiency, generation fidelity, hallucination risk and contradiction. Weak findings produce an evidence gap, which becomes a request for more media — and the loop closes when the inspector captures it.
  6. Verify — a human accepts, modifies, rejects, or asks for more. Only verified decisions enter the package.
  7. Preserve — Evidence Reel, PDF report, and a provenance manifest, all in B2, all traceable to source.

The distinction the whole system exists to hold

Generation creates a candidate. Verification creates evidence.

Nothing generated is ever presented as fact. A candidate carries the model's confidence and its evidence strength as separate numbers, and the interface says plainly: these are machine measurements; nothing here is verified until you decide it is.

AI providers and models used

Role Provider Model Purpose
Media understanding OpenAI (via genblaze-openai) gpt-4o-2024-08-06 Vision analysis of extracted frames → candidate findings
Image generation OpenAI (via genblaze-openai) gpt-image-1 (DalleProvider) Evidence crops, annotated visuals
Video generation OpenAI (via genblaze-openai) sora-2 (SoraProvider) Evidence Reel segments
Offline / degraded Genblaze MockProvider Deterministic fallback — see below

Configured as package:ClassName:model, so any Genblaze adapter can be substituted without code changes. Verified alternatives, read from the installed packages rather than documentation:

Extra Credential Classes chat()?
google GEMINI_API_KEY ImagenProvider, VeoProvider, GeminiImageProvider yes
gmicloud GMI_API_KEY GMICloudImage/Video/AudioProvider yes
nvidia NVIDIA_API_KEY NvidiaImage/Video/Audio/ChatProvider yes
runway RUNWAYML_API_SECRET RunwayProvider no
elevenlabs ELEVENLABS_API_KEY ElevenLabsTTS/SFXProvider no

That last column matters: analysis is a chat() call, so TEXT_PROVIDER needs an adapter that exposes one. The worker checks all three roles at startup and reports provider_ready, provider_unset or provider_unavailable per role.

How it uses Genblaze

Genblaze is the orchestration layer, not a wrapped API call.

  • Generation is a Pipeline, not a request. Each evidence artifact is a Pipeline of Steps with a declared modality, run against a provider resolved from configuration. Swapping OpenAI for Google is an environment variable.
  • Understanding uses chat() with vision-capable models. Frames are sent as image_url blocks, capped at MAX_ANALYSIS_FRAMES=12 because vision calls are billed per image, with response_format={"type":"json_object"} enforcing the shape.
  • The manifest is the provenance root. Genblaze's canonical_hash is stored as the generation's output_hash, so FIELDREEL's evidence graph and Genblaze's own run manifest reference the same verifiable number. genblaze verify can confirm it independently.
  • S3StorageBackend.for_backblaze() is the sink, so generated artifacts land in B2 through Genblaze's own storage path with the manifest alongside.
  • Degradation is honest. When a provider refuses — quota, auth, outage — the run falls through to Genblaze's MockProvider, still a real Pipeline with a verified manifest, and is labelled execution_mode: deterministic with the reason recorded and shown on screen. A degraded result can never be mistaken for a model's opinion. insufficient_quota and rate_limit_exceeded both arrive as 429 and are classified differently, so the queue stops retrying a billing problem it cannot solve.

How it uses Backblaze B2

B2 is the Media Evidence Vault, not a place to put the final file.

Every layer of the media lifecycle is stored and addressable:

orgs/{org}/inspections/{inspection}/
  {source}/{media}/…          original capture, immutable
  {frames}/{media}/…          extracted frames, proxy, thumbnails
  {generated}/{generation}/…  evidence crops, annotated visuals
  {reels}/{reel}/…            Evidence Reel
  {reports}/{report}/…        PDF package

The segment names are configurable (fieldreel.storage.prefixes) so a bucket shared with other tooling can be namespaced; the org/inspection root is not.

  • Direct browser upload via presigned PUT. Inspection video runs to gigabytes; routing it through PHP is the first thing that falls over.
  • The worker holds no bucket credentials. It receives job-scoped presigned URLs for exactly the objects it needs — a TTL sized to the job, not the 15-minute browser TTL, because a URL that expires mid-extraction fails on frame 47 of 60.
  • Every object carries a sha256:-prefixed checksum in a varchar(71) column — exactly the prefix plus 64 hex — and source media is immutable at the database level: a trigger rejects any change to a source asset's checksum once set.
  • Provenance is a graph, not a log. Each artifact records its source asset, the finding it supports, the generation that produced it, the model used, its evaluation, and who verified it.

How we built it

Laravel 11 (system of record, evidence engine, provenance), Python/FastAPI (media worker, Genblaze integration), Next.js 14 (verification interface), PostgreSQL, Redis-backed priority queues.

Internal calls are HMAC-signed over {timestamp}.{nonce}.{sha256 of body} in both directions, with nonce replay protection.

Fourteen static checks run in CI, each written after something broke: schema ordering, unknown columns, mass assignment, model defaults, digest column widths, public-id boundaries, enum cases, eager loading, method declarations and calls, class imports, queue coverage, report block coverage, API↔UI field parity, API↔worker signing parity, and settings usage. Every one has a negative test proving it catches the defect it was written for.

Challenges

Everything that broke, broke in execution — never in review. Roughly twenty defects surfaced this way: a queue worker consuming the default queue while every job was dispatched to named priority queues; a signature verifier using a sha256:-prefixed digest while the signer used the bare hex, so every callback 401'd; a Content-Type header the presigned URL never signed, which AWS ignores and B2 answers 500 to; rstrip("/x") stripping a character set and leaving /./ in every frame key, which GPT-4o then could not download.

The most instructive one: an idempotency key that did not include the media. The first analysis ran against broken frame URLs, produced nothing, and cached that empty result — so "Run analysis" succeeded and did nothing, forever. A no-op that reports success is worse than a crash.

What we learned

The checks are the durable artifact. Each encodes a specific surprise so it cannot recur, and each was written only after the surprise happened — which is also why several early ones passed vacuously until negative-tested.

What's next for FIELDREEL

Longitudinal inspection: the same asset over time, with progression tracking and recurring/resolved findings. The pipeline already stores everything needed.

Built With

  • evidence-engine
  • genblaze-integration)
  • laravel-12-(system-of-record
  • next.js-14-(verification-interface)
  • postgresql
  • provenance)
  • python/fastapi-(media-worker
  • redis-backed
Share this project:

Updates

Submission history