Inspiration

We walked past tons of static AI startup billboards per day in San Francisco. They just display pre-defined sets of posters from YC, Fetch.ai, Convex, etc. randomly. But how many of them are _ actually _ getting seen? So, why don't we make this billboard smarter and more strategic?

What it does

We are here to revolutionize billboards. Meet BillboardOS.

BillboardOS is the first AI-native and agent-to-agent billboard that automatically selects and plays the right ad videos for sponsoring brands.

First, a webcam on the billboard counts how many people are watching. It analyzes how many people were in view, how long they stayed, and how many faced the screen for at least a second. Our agents then find companies that fit that audience and open a live, agent-to-agent auction for the next 30-second slot. The highest bidder pays and goes on air, and while the ad is playing, we keep going for another auction. No humans in the loop, just automated agent-to-agent negotiation from bidding to display. => We strongly believe that this is the future of marketing and automated, more personalized billboards.

How we built it

  • Convex is our choice for the backend: database, server functions, scheduled jobs, and real-time sync, so the screen, the host console, and every advertiser dashboard update together.
  • Coinbase CDP + x402 handles agent-to-agent payments: the winning agent pays in USDC on the Base Sepolia testnet, and an ad only goes on air once there's a successful transaction in real time. Architecture: Buyer agent (CDP wallet) → HTTP 402 challenge (Convex site / x402 resource) → signed payment retry → CDP Facilitator verify + settle (Base Sepolia) → Convex booking + playback receipt Host wallet receives USDC; Convex enforces holds, idempotency, and no double-pay.
  • Creao.ai agents supervise the whole sales loop: they do extensive research for different companies to choose which companies fit a slot, shortlist buyers for the auction, and keep tuning the coefficients in our base-bid price model so the opening ask stays sharp => the mastermind behind quick flash deals.
  • Bitdeer AI Model Studio models run inside the live auction. It’s a fast model that sets the base bid and negotiates agent-to-agent with each company’s buying agents to push for the best price without leaving either side underwater in only 30 seconds for the schedule on air.
  • MediaPipe in the browser: EfficientDet-Lite0 finds people, our own tracker follows each person across frames so they're counted once, and Face Landmarker's head direction tells us who is facing the screen.
  • The price model is a ridge regression retrained every hour on every closed window.
  • CREAO agents research each advertiser to estimate brand value and plan sales.
  • React, Vite, and Tailwind power the screen, the host console, and the advertiser dashboard, and Convex hosts the site too.

CREAO

  • Track: Best One-Person Company / Best AI-First Disruptor
  • Sales Agent ID: 82fa549a-99bd-402d-9990-4124d9f97cab
  • Revenue Agent ID: dc1b236c-1e3e-4db6-9134-7f406f6374d5
  • Example run: https://agent.creao.ai/share/53777b53b451c52b8adeb850770b3ec2
  • Role: CREAO qualifies buyers / plans revenue; Bitdeer bids in the 30s window; Convex settles after x402 pay.

Challenges we ran into

  • Agent-to-agent communication This is the unique feature that we want to bring into this product. One of the biggest challenges we ran into was ensuring smooth and fluid communication between the billboard’s agents and the buyer companies’ agents. There are a lot of interactions between them going on, such as bidding on price, negotiating the base bid, finding the winners, and deciding the strategy for the next round, and every model has to talk to the others clearly and fast. Finding that sweet spot between efficiency and speed was definitely our challenging part, and we had to carefully choose which model supervises and which one does the actual work.
  • ** Payment reliability** Payments, holds, and 30-second windows race each other. Every step is a transaction; payments can't be applied twice, a late or wrong payment becomes a finance exception, etc. We have to consider the payment carefully to make sure every transaction is valid.
  • Face Detection and Counting Model Our first counter added a person every time the number of boxes went up, so someone stepping out of frame and back counted twice. We wrote a tracker that follows each person instead. The browser's GPU mode also returned junk detections for our model, so we have to switch to CPU to fix it.
  • Proof vs. privacy. Advertisers want proof, and passers-by don't want to be filmed. All video stays on the device; only counts and anonymous boxes are sent.
  • Moving fast as a team. With several of us shipping to one shared backend, the website and the server functions kept drifting apart, so we learned to deploy backend changes first.

Accomplishments that we're proud of

We are proud that we have discovered a new business venture for the future of marketing and billboards, specifically. It would no longer require humans in the loop, and everything is going to be agent communication. => This would significantly increase efficiency and help us be involved in more meaningful and touching work of marketing.

  • A screen that runs a full find-buyer, price, get-paid, play loop every 30 seconds with no human involved.
  • Real on-device people counting with no video ever leaving the screen.
  • Real agent-to-agent payments over x402 on testnet.

What we learned

  • Agents are great and smart negotiators, so learning from their interaction is pretty fun. However, we also have to emphasize governing the interaction and making sure it is safe. This again shows the importance of AI Alignment and Safety in our era. However, money needs hard rules: agents propose, the database decides.
  • A small model running on the device beats a big model in the cloud for privacy and speed, but where you point the camera matters more than which model you use.

What's next

There's a lot more to explore with this new form of marketing. We have thought of some potential ways:

  • Pilot on real screens at a venue with real advertisers.
  • Mainnet payments and a real bank feed (Capital One Nessie).
  • Better attention measurement and multi-screen packages for advertisers.

Built With

Share this project:

Updates

Submission history