Advertisement-Feature-Extraction

This is a submission for the AppLovin Ad Intelligence challenge where we extract insightful features. We use OpenRouter's Andromeda, Claude, Deepgram, and YAMNet.

Comment

Ad Intelligence Challenge CalHacks 2025

How to run

Make sure you have uv (package manager + venv) installed! \ To run the pipeline on the given dataset: 

cd CalHacks25-Ad-Intelligence
uv run python main.py

Architecture

To get to high level features, we start with extracting base properties of ads using computer vision and audio processing. We gather scene details, text content, speech, audio events, an visual aesthetics. Depending on image vs video format, we perform analysis accordingly. For videos, we concurrently extract speed and auddio events using Deepgram and YAMNet, while gathering visual features from a subset of frames using VLMs from OpenRouter. Once all are gathered, we aggregate them to a large reasoning model to derive high level features leveraging the model's intuition of brand history and trends. Video:

Features

Feature Data type Justification
Transaction-focused Sales Approach Integer from 1-5 Studies show that approaches which emphasize the product or communicate an urgency to buy the product ultimately have negative influences on likes, comments, and shares.
Relationship-focused Sales Approach Integer from 1-5 In this approach, sellers attempt to create an emotional bond with the customer by sharing their private lives and treating the customer as a friend ("Come with me to this niche restaurant!"). While less research has been done on the impacts of this feature, emotions are generally correlated with consumer engagement–we expect to have increased engagement and clickthrough rates if a customer feels connected to the seller.
Political Leaning String This is helpful for determining the correct demographic to recommend the ad to. For example, ads embracing diversity values may perform better with a more progressive audience.
Controversy Integer from 1-5 Controversial ads may drive engagement rates through the mention of "taboo" topics, such as dark or sexual content.
Emotions String Different emotions can elicit a variety of responses, which can predict engagement rates. For example, inducing pity in a customer may make them more likely to purchase a product.
Visual Intensity Integer from 1-5 More visually intense ads may be more captivating or attention grabbing, boosting retention.
Storytelling Boolean Using a story or narrative can help viewers remember the product.
Seasonality/timing String We want to ensure that the customer is viewing the ad at the "right" time. This can apply to ads which promote seasonal products, such as winter jackets or pumpkin spice lattes.
Brand Consistency Integer from 1-5 How much the brand adheres to the visual and marketing techniques based on the brand's industry and identity. This can help strengthen assocation to a product, and make a customer more likely to buy in the future.
Category String This is the general category that the brand falls under, such as fashion, food, or health. This can tailor recommendations for customers who have historically expressed interest in the category.
Creativity Integer from 1-5 This measures how novel and original of a concept the ad uses. The story may be unique, or they may use interesting visual elements. Customers might be more likely to click away if they recognize the ad as an ad (such as sponsor segments on YouTube), so creativity can help boost engagement.
Call To Action Integer from 1-5 A call to action, such as a button or text saying "Buy Now!" can help clickthrough rates. Additionally, this can double as a measurement for the interactivity of the ad; mobile games often prompt the viewer to take an action in the context of their game, which can spur the viewer to download the game.
Demographic Focus String The age demographic that this ad is targeted towards, such as Gen-Z, Millennials, etc. Different demographics have different cultural values, which the ad may be targeting.
Cultural Relevance Integer from 1-5 Related to the age demographic, this describes how well the ad ties into their associated cultural values. For example, Gen-Z and Gen-Alpha may be more likely to appreciate TikTok memes than boomers.
Target Audience String This feature can determine the demographics that this ad would resonate the most with.
Least Fitting Audience String This feature can determine what audiences this ad should generally NOT be shown to.

Built With

  • claude
  • deepgram
  • openrouter
  • python
  • thinking
  • vlm
  • yamnet
Share this project:

Updates