Inspiration
Creators spend hours producing short-form content, but most of that content only gets monetized when a sponsorship was arranged before it was filmed. That leaves creators with entire back catalogs of videos that may still be generating views but have no way to generate new sponsorship revenue.
Traditional brand deals also create more work. Creators have to negotiate an integration, script it, film with the product, and often edit a dedicated sponsored segment into their content.
We built SceneSponsor around a different idea: what if the content a creator has already made could become advertising inventory?
SceneSponsor uses AI to find natural places for products inside existing short-form videos. A coffee cup can sit on an empty desk, a shopping bag can appear on a counter, or a poster can occupy unused wall space.
Instead of asking creators to make more ads, we want to help them monetize the hours of content they already have.
What it does
SceneSponsor analyzes an existing short-form video and searches for locations where a sponsored product could naturally fit.
We currently focus on several types of placements:
- cups, cans, glasses, and bottles on desks, tables, or counters, including opportunities to replace an existing generic drink
- posters on suitable blank walls
- tote bags in plausible hanging locations
- shopping bags on empty surfaces
The system reasons about both space and time. It identifies where a product could physically fit in the frame and during which portion of the video that placement remains viable.
We visualize this through three synchronized versions of the Reel shown side-by-side:
Original → AI Placement → Sponsored Result
The first shows the untouched content. The second exposes the AI's reasoning by drawing boxes over the areas it identifies as potential product-placement inventory. The third shows what the video could look like with a product placed into the scene.
This makes the AI's decision visible instead of treating product placement as a black box.
How we built it
We built the interface with Next.js, React, and TypeScript.
The core of SceneSponsor is the video-analysis pipeline powered by Fireworks AI.
We sample the source Reel at 4 frames per second so the model can reason about rapidly changing camera angles and scenes. Rather than analyzing isolated images, we analyze groups of consecutive frames so the model has temporal context.
For each section of the video, the model evaluates whether any of our supported product categories have a genuinely plausible placement. We specifically instruct it not to force a placement when one does not make sense.
For each opportunity, the analysis produces structured information including:
- product category
- whether the product should be added or replace an existing object
- start and end timestamps
- normalized bounding-box coordinates
- a physical anchor point for the product
- confidence and occlusion risk
- a description of the surface or location
Because the coordinates are normalized, the frontend can map them directly onto the video regardless of its displayed size.
We then use the video's current timestamp to determine which placements should be visible and interpolate between analyzed positions to keep the boxes moving smoothly between sampled frames.
The three videos are synchronized so the viewer can directly compare the original footage, the AI's placement decision, and the resulting sponsored version at the same moment in time.
Challenges we ran into
The main challenge was realizing that this is not simply an image-generation problem.
Finding an empty area in a single frame is relatively easy. Finding an area where a product can physically and naturally exist across multiple frames is much harder.
Camera angles change. People move in front of surfaces. Hands cross desks and counters. An area that looks perfect for a coffee cup at one moment may disappear half a second later.
That led us to analyze video at 4 FPS and give the model groups of consecutive frames rather than asking it to make independent decisions from individual screenshots.
We also had to distinguish between an area being technically empty and actually being a good product placement. A useful placement needs appropriate scale, a plausible supporting surface, sufficient visibility, low occlusion, and enough duration to be meaningful.
Another challenge was visualization. We wanted someone watching the demo to understand what the AI was doing immediately, which led to the synchronized three-Reel interface rather than hiding the analysis behind a loading screen.
Accomplishments that we're proud of
The part we're most interested in is that SceneSponsor does more than detect objects that are already in a video. It reasons about what could plausibly be added to the scene.
For example, it can distinguish between an empty section of desk that could hold a beverage and a blank wall that could support a poster. It can also identify an existing generic drink as a potential replacement opportunity rather than unnecessarily adding another object.
We also made the model's reasoning visible. The middle Reel turns its output into temporal bounding boxes, so you can watch potential advertising inventory appear and disappear as the scene changes.
That makes the core concept easy to understand:
Existing content can contain inventory that was never monetized when the video was originally created.
What we learned
We started with the idea of using AI to insert products into videos, but the more interesting problem turned out to be deciding where and when a product should exist in the first place.
A convincing placement has to understand the physical scene. A cup needs a surface. A poster needs a wall. A shopping bag needs enough space to realistically occupy. The placement also needs to survive long enough across the video to be useful.
We also learned that the system should be willing to say there is no good placement. Forcing every product category into every video produces worse results than identifying a small number of high-quality opportunities.
More broadly, we realized that creator content can be thought of as having previously unused inventory. The empty desk, wall, shelf, or counter in an existing video can potentially become monetizable without requiring the creator to reshoot the content.
What's next for SceneSponsor
The next technical challenge is making the final product insertion as robust as the placement analysis.
We want to improve tracking across camera movement, handle objects passing in front of sponsored products, and better preserve perspective, lighting, shadows, and scale so inserted products remain convincing throughout a shot.
Beyond the technical pipeline, the larger opportunity is applying SceneSponsor to creators' existing content libraries.
A creator could connect a catalog of previously published videos, let SceneSponsor identify the available placement inventory across that catalog, and make suitable opportunities available to brands.
Instead of every sponsorship requiring another shoot, creators could potentially generate new revenue from videos they have already made.
Longer term, we see SceneSponsor as infrastructure for programmatic product placement: matching brands with appropriate moments inside creator content while turning hours of previously unmonetized content into new inventory.
Log in or sign up for Devpost to join the conversation.