Reverse Push

Inspiration

Creative teams can see a strong image, GIF, or motion reference but still struggle to translate it into a prompt precise enough to reproduce. Manual analysis misses typography, layout zones, lighting, materials, camera behavior, and timing. Reverse Push turns visual references into structured, reusable generation instructions directly from the browser.

What it does

Reverse Push is a Chrome Manifest V3 extension for reverse-prompting visual media. A user can right-click an image or video, capture a page region or motion area, or upload a local image/video. The extension then:

analyzes static images, GIFs, and sampled video frames;
extracts composition, visible text, color, lighting, materials, subject details, camera behavior, motion, and timeline anchors;
produces concise Chinese and English prompts ready for image or video generation;
adds implementation-oriented motion specifications for recreating animated references;
stores analysis history and reusable visual DNA;
supports fast, balanced, and high-fidelity analysis modes.

How we built it

The product is a browser-native pipeline rather than a single model call. Chrome context menus, screen capture, an offscreen document, and local uploads collect the source media. Motion inputs are sampled across time, with representative frames and a contact sheet sent to the multimodal reasoning layer.

A structured visual contract guides GPT-5.6 to return predictable JSON containing generation prompts, motion timelines, implementation details, and restrictions. The response then passes through parsing, safety cleanup, prompt-quality gates, and fidelity scoring before it is shown to the user.

The model transport supports OpenAI-compatible APIs and an optional local Codex CLI mode. Because Chrome extensions cannot execute local commands directly, we built a loopback-only gateway on 127.0.0.1 that converts the extension's visual request into a sandboxed, ephemeral codex exec task. The gateway accepts only local extension requests, processes one model job at a time, and removes temporary media after each request.

Codex was used throughout Build Week to inspect the existing extension architecture, trace the real request and authentication path, implement the local gateway and dual transport, refactor the UI, strengthen the visual prompt contract, and repeatedly verify the image/video analysis flow.

Challenges we ran into

Preserving visual fidelity

Generic vision descriptions often omit exact text, brand lockups, spatial ratios, light paths, or small but important elements. We introduced a source-fidelity fingerprint and deterministic post-model checks so the final prompt remains anchored to observable evidence.

Understanding motion efficiently

Sending every video frame is expensive and redundant, while sparse sampling can miss key transitions. We implemented uniform temporal sampling, frame metadata, high-detail anchor frames, and contact sheets to balance coverage and cost.

Connecting a browser extension to Codex CLI

A browser extension cannot safely spawn local processes. The local loopback gateway provides a narrow bridge with origin checks, read-only and ephemeral Codex execution, serialized requests, and automatic cleanup.

Keeping model output usable

Multimodal responses can be verbose, malformed, or inconsistent. A schema-oriented contract, JSON recovery, display-safety filters, and quality gates make the result stable enough for direct creative use.

Accomplishments that we're proud of

One workflow for webpage images, videos, GIFs, screen regions, and local uploads.
Bilingual prompts that are ready to reuse instead of raw visual-analysis notes.
Motion-aware output with timelines, keyframes, camera movement, transitions, and implementation guidance.
A dual model path: OpenAI-compatible API or local Codex CLI.
Deterministic fidelity and quality checks layered on top of multimodal reasoning.
A privacy-conscious local gateway that never exposes a LAN service.

What we learned

High-quality reverse prompting is not just an image-captioning problem. The strongest results combine multimodal reasoning with explicit evidence contracts, temporal sampling, deterministic validation, and a user interface that exposes useful creative structure without exposing internal debugging noise.

We also learned that local AI tooling needs carefully designed boundaries. The smallest secure bridge between a browser and a local CLI is more reliable than trying to hide process execution inside the extension.