Showrunner

The agent that tasks the humans: it ingests every guest photo at a live event, spots coverage gaps, dispatches photo bounties to guests' phones, and premieres AI-edited reels the same night.

Comment

Inspiration (Bring Your Own Friction)

Every wedding, trip, and milestone celebration ends with the same chore: photos scattered across dozens of guest phones, endless group chat requests ("can you send me that one?"), and weeks waiting for an album where key people were barely photographed.

Last December at my cousin's wedding, 500 guests took over 4,000 photos, but the couple received their album five months later, and key moments (like the groom's mother during the Haldi ceremony) were completely missed. Traditional event apps are passive shared folders that force humans to manually collect, organize, and edit.

Showrunner flips this paradigm entirely: an autonomous agent fleet that directs an event's media in real time. When photo coverage is missing, the agent doesn't wait for humans — it tasks them.

What It Does

Showrunner operates an autonomous data and control loop with zero chat windows:

  1. Zero-Friction Guest Uploads: Guests scan a QR code on their phone (anonymous PWA, no app store download, no account required) and upload photos.
  2. Perception Fleet: Within seconds, background workers score each shot against the event schedule and cultural glossary (Gemini 3.5 Flash-Lite), index faces into 512-d embeddings via InsightFace + Firestore Vector Search, and filter for safety (Cloud Vision SafeSearch + Gemini dignity rubric).
  3. Autonomous Live Wall (Kiosk): A big screen (/kiosk/[eventId]) displays dynamically ranked highlights, live wanted posters for missing shots, and premiere recap reels.
  4. Story Director & Bounties: Every 2 minutes on Cloud Scheduler, the Story Director (Gemini 3.7 Flash) reconciles the timeline against photo coverage. If a required moment or VIP is missing, it dispatches targeted photo bounties to guests' phones with point rewards.
  5. Reel Director: The Reel Director (Gemini 3.7 Flash) plans vertical video reels, writes narrative briefs, orchestrates original AI soundtracks via Lyria 3, prepends cinematic openers via Veo 3.1 Fast, and compiles them with beat-synced transitions via Cloud Run Jobs (FFmpeg + Librosa).
  6. Private Guest Albums: Guests take an optional selfie to unlock a private album containing every photo they appear in across the entire event.

How We Built It & The Model Fleet

AI Models & Multi-Model Collaboration:

  • Gemini 3.7 Flash (gemini-3.7-flash): Drives the control plane. Powers the Story Director (coverage ledger reasoning and bounty creation), the Reel Director (narrative storyboarding and edit decision lists), and the Onboarding Itinerary Extractor (zero-shot multi-modal extraction converting raw pastes, PDFs, or photos of itineraries into structured timelines).
  • Gemini 3.5 Flash-Lite (gemini-3.5-flash-lite): Powers the high-volume data plane. Evaluates visual evidence per stage, aesthetics ($0.0–1.0$), and captions on incoming photos (Curator Agent), judges dignity and emotional context (Guardian Agent), and validates storyboards (Reel Critic).
  • Lyria 3 (lyria-3-clip-preview): Composes bespoke 30-second instrumental soundtracks for vertical highlight reels and ambient kiosk audio via the Google GenAI Interactions API, beat-matched using Librosa.
  • Veo 3.1 Fast (veo-3.1-fast-generate-001 via Vertex AI): Generates 8-second cinematic portrait openers from top couple photos.
  • Gemma 4 (gemma-4-26b-a4b-it): Generates private, 3-sentence explainable taste summaries for guests based on their album reactions off the critical path.
  • InsightFace & Firestore Vector Search: Dedicated deterministic 512-d ArcFace ONNX embeddings for biometric face matching. Kept strictly separate from LLMs for security and consent.
  • Google Model Armor & Cloud Vision SafeSearch: Dual-layer governance protecting all text ingress surfaces and enforcing content safety floors.

Architecture & Google Cloud Infrastructure:

  • Google Agent Development Kit (ADK v2 Python) & GenAI SDK (enterprise=True).
  • Cloud Run (8 microservices + 1 Cloud Run Job for 8-vCPU FFmpeg rendering).
  • Cloud Tasks (6 rate-limited queues metering Gemini API spend and prioritizing bounty submissions).
  • Cloud Storage & Eventarc (Direct-to-GCS signed PUT uploads triggering async perception pipelines).
  • Cloud Scheduler (2-minute autonomous director tick loop).
  • Firestore & Firebase (Real-time snapshot synchronization, native Vector Search, and Firebase Hosting).

Challenges We Ran Into

  • Balancing Autonomy with Spend & Rate Limits: Live photo spikes could trigger API rate limits or unbounded costs. We solved this by using rate-limited Cloud Tasks queues (8 dispatches/s) and bounding token usage with low-resolution image tiers (~258 tokens/photo).
  • Avoiding Context Rot: Event data accumulates rapidly. Instead of letting agent prompts grow indefinitely, the Story Director operates over a fixed 10-tick rolling session window compacted through deterministic arithmetic.
  • Deterministic Trust vs. LLM Hallucinations: LLMs propose actions, but an entirely deterministic validator (act.py) and transactional visibility function (recompute_visibility) enforce point caps, consent, and exposure. No LLM ever directly gates public visibility.

Accomplishments We're Proud Of

  • True Unsupervised Autonomy: An end-to-end background workflow that executes without human prompts, buttons, or chat interfaces.
  • Multi-Modal Synchronization: Directing AI music generation (Lyria), video generation (Veo), multimodal perception (Gemini 3.5), and complex reasoning (Gemini 3.7) in a single synchronized production pipeline.
  • Fast End-to-End Latency: Photos progress from upload to classification, face indexing, safety screening, and live wall display in under 6 seconds.
  • Rigorous Governance & Consent: 3-ring consent model with retroactive photo retraction and instant wall removal in under 5 seconds.

What We Learned

  • Separating probabilistic LLM reasoning (opinions) from deterministic policy enforcement (visibility and points) is essential for production-grade agentic applications.
  • Synchronizing AI-generated music (Lyria) with generative video cuts requires real-world audio analysis (Librosa beat tracking) rather than assuming model prompts will perfectly match timing.

What's Next

  • Multi-camera live-stream feed ingestion with automated real-time clip generation.
  • Dynamic multi-room kiosk syncing across large venue networks.
  • Export integrations for professional photographers (Lightroom / Capture One catalog sync).

Built With

Share this project:

Updates

Submission history