Lumen - Audience of ONE

A privacy-first engine bridging the physical-digital divide. Using Gemini 3, it serves bespoke content and a generative virtual try-on to any group or individual—an anonymous Audience of ONE.

Comment

Project Lumen: The Audience of One

Inspiration

As a veteran of the digital signage industry, I’ve seen thousands of screens across the globe suffer the same fate: they become expensive digital wallpaper. Once the initial excitement of an installation fades, the "Management Tax" takes over. Content goes stale, administrators lose interest, and screens that should be dynamic end up stuck in time—I've seen signs still displaying COVID-19 protocols in 2026.

We built Project Lumen to kill the "management" model and replace it with an "autonomous" one. We wanted a sign that doesn't just loop video, but actually sees and reasons. We were inspired to create a living concierge—a sign that understands the context of the room and audience present, and instantly transforms into an experience, allowing users to virtually step into the products they see.

What it does

Lumen transforms digital screens into a responsive participant. At its core is a "Heartbeat" powered by Gemini 3 Flash. It senses the environment—identifying the audience—and instantly generates a bespoke visual content tailored to that specific moment with a product from a preset catalog (product placement).

But that is only half the battle. With a single tap, Lumen enters Smart Mirror Mode. Users don't just look at a product; they wear it. Utilizing high-fidelity generative AI, users can select an item and see it realistically mapped to their own reflection. It is the world’s first complete retail funnel—from anonymous discovery to interactive virtual try-on—on a single, privacy-first screen.

How we built it

  • The Eyes (camera attached to the screen):Snapping pictures at a preset adjustable intervals**
  • The Brain (Gemini 3 Flash): We utilized Gemini 3 Flash for high-speed multimodal reasoning. It doesn't just "detect" objects; it reasons about the context of the audience to decide what to show.
  • The Creator (Gemini 3 Pro): We pushed the boundaries of image generation to create high-fidelity signage assets and personalized overlays using a product from the catalog.
  • The Engine (Firebase): A serverless architecture using Cloud Functions keeps our API keys secure and our local hardware footprint near zero.

Challenges we ran into

So many challenges, but the biggest hurdle was the "Latency Gap." By placing an external camera, the targeted audience can be detected 10-15 seconds away and the new content will be on the screen by the time they get there.

  1. Hardware Sync: We had to engineer a "Lens Guard" in Dart to perfectly time the 2000ms delay required for hardware shutters to initialize without freezing the UI.
  2. Multimodal Orchestration: Coordinating the "Reasoning" (Flash) and the "Generation" (Imagen) to happen in parallel was a complex state-management challenge, solved by creating a centralized "Heartbeat" controller.
  3. Spatial Accuracy: For the Virtual Try-On, getting the AI to place products accurately required precise prompt engineering and coordinate mapping within the Gemini vision context.

Accomplishments that we're proud of

We successfully achieved a full multimodal round-trip—sensing, reasoning, and generating—in a rapid, automated cycle. We are also incredibly proud of our Privacy-First "Forgetful" Architecture. We proved that you can deliver extreme personalization without ever storing a single byte of PII or a single user image; Lumen sees, reacts, and immediately deletes the data.

What we learned

We learned that Context > Identity. The industry is obsessed with tracking "who" a person is, but we found that understanding "what" the moment needs is far more powerful. We also discovered that Gemini 3 is uniquely capable of switching between high-level reasoning and granular spatial mapping, especially when utilizing "low" thinking levels for faster analysis.

What's next for Lumen

We plan to integrate ** Local Models*, for faster processing and **Real-Time Inventory Awareness*, allowing Lumen to only suggest products that are currently in stock. We are moving toward a world where every screen is a gateway to a personalized, autonomous boutique—turning "digital wallpaper" into a living, breathing experience. The "Audience of One" for digital signage is just the beginning; imagine extending this technology to all types of content from movies, song video clips to YouTube videos!!! The sky is the limit.

Built With

Share this project:

Updates