Ad-Lab

Landing Page
Simulation
Agent Interface
Competitor Analysis

Inspiration

DTC brands usually find out which ads work by spending money on them. They make a few variations, run them on Meta, wait for clicks and purchases, and try to figure out what worked. But a lot of the answers to what customers care about are already out there, buried in reviews, Reddit threads, social media posts, and competitor ads.

We wanted to try a different approach: simulate how different customers might respond to ads before spending money on real campaigns. Instead of treating customer feedback as something to read after a campaign, we use it to build simulated shoppers, test ad ideas against them, and learn which messages resonate with different customer segments.

What it does

Ad Lab uses four AI agents to research a brand, generate ad ideas, and run a simulation of customer behaviour. You send one message in ASI:One, "Find ads that can beat Magic Spoon's current ones", and the agents take it from there.

Scout collects public customer feedback through Apify. It looks at Magic Spoon mentions across Reddit, Trustpilot, Instagram, TikTok, and X, along with Meta ads from seven competitors: Catalina Crunch, Seven Sundays, Surreal, GHOST, Premier Protein, Legendary Foods, and Quest. It then builds a brand profile with customer preferences, complaints, and objections, backed by real quotes.
Strategist uses that research to come up with ad recipes, each made up of an angle and a hook. Some ideas are inspired by competitor ads, some build on Magic Spoon's existing ads, and others focus on customer concerns that competitors aren't addressing. Magic Spoon's current ads serve as the control.
The simulated market contains 300 shoppers across five segments built from real reviews and social posts. Each shopper has a profile and memory. Every simulated day, they see one of our ads between two real competitor ads in a feed. TypeSafe's Jev model reads the shopper's profile and memory and estimates the probability that they click the ad, stay on the landing page, and buy. Our code samples the outcome, applies the budget rules, and updates what the shopper remembers.
Thompson sampling adjusts how impressions are distributed across ad recipes. Recipes that perform better in a particular segment get more impressions, while at least 10% of impressions remain on the control. The system also records learnings from the simulation, such as whether addressing concerns about taste early seems to work better for a particular segment.
Copywriter uses those learnings to write new ad variations for the next round. It chooses which insights to apply, generates the copy, and checks numerical claims against verified brand facts.
Ad Lab posts a leaderboard after each round and reports the top-performing ads for each segment at the end.

The point is to simulate customer behaviour in a controlled environment, not to claim that simulated shoppers are real customers. We use the results to compare creative ideas and decide which ones might be worth testing with real ad spend.

Every agent message, Apify job, and simulated decision is stored in SpacetimeDB, so the entire process is visible in the web app:

/agents shows agent conversations, scraping jobs, and the task board.
/sim shows how the bandit reallocates impressions, leaderboards for each segment, a live feed of shopper decisions, and the learnings generated by each round. You can pin a shopper and inspect the exact text Jev received. A "what if" toggle removes a line from their memory and runs the model again to see how its response changes.
Brand Identity and Competitor Analysis show the research gathered from real customer feedback and competitor ads.

How we built it

Data (Apify + Supabase). We collected 454 brand mentions across five sources and 239 Meta ads across Magic Spoon and its competitors. For Meta ads, we used advertiser page IDs instead of keyword search because keyword search returned too many unrelated advertisers.

We labelled the ads by angle, hook, tone, and format, then embedded them for analysis. The pipeline distilled the data into 96 brand facts: positioning and proof points checked against sentences on magicspoon.com, 38 customer-voice facts containing 85 verbatim quotes, 21 market priors derived from 51 competitor ads selected as winners in our analysis, and grounded shopper personas.

This gave us five shopper segments, eleven ad recipes plus a control, and 144 real ads to use in the simulated feeds.

Agents (Fetch.ai). Our four Python uAgents are registered on Agentverse. They communicate through typed protocols, and each also supports the chat protocol so judges can message them directly in ASI:One. Ad Lab responds with Interactive Cards showing the brand profile, competitor findings, recipes, leaderboards, and winning ads.

The agents call a Next.js HTTP API to start the Supabase job queue, build recipes, and generate creatives.

Shared state (SpacetimeDB). One TypeScript module holds the system's shared state. We were a two-person team, so we split the work around a written data contract. One side handles segments, recipes, and creatives, while the other handles simulation runs, decisions, statistics, and learnings. A task board connects the two sides between rounds.

A single claim_task reducer ensures two workers cannot claim the same task. The web app subscribes directly to the tables, so updates appear live without polling.

Simulation (TypeScript + Jev). Each shopper's profile and memory are converted into plain text under strict rules: fewer than 450 words, no repeated facts, numbers written out in words, and explicit statements when the shopper has no prior opinion of Magic Spoon. We also keep the budget out of the model's input and enforce it in code.

We calibrate Jev's raw probabilities during a warm-up round by fitting a log-odds offset b for each outcome. This brings the simulated rates closer to broad industry ranges: roughly 1% tap-through, 50% landing-page stay, and 4% purchase after staying.

$$\hat{p} = \sigma\left(\operatorname{logit}(p) + b\right)$$

Since actual taps are rare at a 1% tap rate, they aren't very useful for ranking ads in a short simulation. Instead, we score each impression using its expected number of orders:

$$r = P(\text{tap}) \times P(\text{stay}) \times P(\text{buy} \mid \text{stay}) \times \text{budget guard}$$

The bandit uses these scores to allocate impressions within each segment, shares evidence across segments, and keeps at least 10% of impressions on the control.

Every Jev response is cached using a SHA-256 hash of the exact request. This means a completed run can be replayed with identical results without calling the model again. A full run contains around 8,700 decisions and costs roughly $0.33 in Jev calls.

Creatives. We built on our existing ad-generation pipeline. Each brief combines a shopper persona with an ad angle, and every numerical claim in the generated copy must be supported by a verified brand fact. Learnings reach the copywriter as number-free directives, so simulated performance numbers cannot accidentally become claims in the ads.

We generate images for the top ads with gpt-image-2 using the brand's product photo.

Frontend. Next.js 16, React 19, and Tailwind CSS 4.

Challenges we ran into

Prompt length. Our first prompts built from real data reached 707 words, but the simulation limits shopper states to 450 words because too much irrelevant context hurts consistency. We shortened ad descriptions, kept only the text visible before "more", and removed duplicate quotes across segments.
Keeping real and simulated knowledge separate. Some customer quotes mentioned brands that a simulated shopper was supposed to know nothing about. We removed brand names where necessary to keep the shopper's knowledge consistent. We also made sure simulated performance numbers couldn't leak into ad copy.
Grounding quotes correctly. Initially, our data pipeline attached the first quote from a review to every theme mentioned in that review. As a result, only 48 of 99 quotes actually matched their assigned themes. We fixed this by selecting the sentence containing the relevant phrase.
Position bias. In early tests, asking the model to choose one of three ads made its answer sensitive to the order of the options. We switched to asking one yes/no question per ad while listing all three ads in the shopper's state.
Multiple agents sharing one database. The agents, simulation worker, and fallback creative worker all update shared tables. We isolated each run in its own database, added a 60-second fallback if Copywriter doesn't claim a task, and built a seeded test environment that resets in about 30 seconds.

Accomplishments that we're proud of

The entire loop runs from one message. A single ASI:One message can trigger scraping, data analysis, planning, simulation, rewriting, and reporting. Our headless end-to-end test passes all 11 checks, with all 43 expected agent messages logged.
Every decision is inspectable. Quotes can be checked against the original source text. Each simulated decision stores the exact text Jev received, its raw and calibrated probabilities, and the random outcome.
Runs are reproducible. Cached Jev responses and seeded simulation logic let us replay a completed run with identical results and no additional model calls.
Two people built separate parts in parallel. We worked against a shared data contract and integrated the two sides through the task board and database.

What we learned

Use the model to estimate behaviour, not control the simulation. Jev estimates how a shopper might respond. Code handles the random outcomes, budget limits, memory updates, and impression allocation. This makes the simulation easier to reproduce and inspect.
Calibration matters. Raw model probabilities can be too high, so we calibrate them before using them to compare ads.
Shared state makes multi-agent systems easier to debug. Storing every message and decision in SpacetimeDB gives us a live view of what the agents are doing and where a run might have gone wrong.
Grounding needs actual checks. We validate quotes against source text and numerical claims against brand facts instead of relying on the model to follow instructions.

What's next for Ad Lab

Compare the simulation with real ad spend. Run the strongest recipes on Meta with small budgets and calibrate the simulation against observed click-through and conversion rates.
Support any brand with less setup. The /start flow already accepts a website and product photo. Next, we want Scout to discover competitors automatically and build a starting profile for a new brand.
Make the simulated shoppers more realistic. Add image-aware ad perception, learnings about ad fatigue from repeated exposure, and additional feeds such as TikTok and YouTube.
Charge per test. Explore charging for simulation runs through Fetch.ai's Payment Protocol.

Ad Lab's results are rankings within a simulation, not forecasts of real campaign performance. The shoppers are built from real customer feedback, and Jev estimates their responses, but the results still depend on the model, calibration, and simulated environment. We want to validate the rankings against controlled real-world ad tests before treating them as evidence of actual campaign performance.