Shine - AI Micro-Drama Video Studio

Inspiration

The vertical micro-drama industry (9:16 Wēi Duǎnjù / Short Drama) has exploded into a $5B+ global entertainment phenomenon across TikTok, YouTube Shorts, and Instagram Reels. Millions of viewers consume 1-to-3 minute cliffhanger-driven episodes daily.

However, creating high-quality vertical drama series has remained bottlenecked by traditional Hollywood-style workflows:

Weeks of Production Time: Scripting, shooting, and editing 30–50 episodes takes traditional studios 3–6 weeks.
Character Continuity Nightmares: Generative video tools frequently suffer from character drift, where the protagonist's face or outfit changes from shot to shot.
Complex Multi-Track Editing: Syncing dialogue, dynamic caption pop-ups, sound effects, and cliffhanger transitions in vertical 9:16 format is tedious and fragmented across different tools.

We asked: What if an autonomous fleet of specialized AI Director agents could take a single high-level premise and produce an entire 30-episode cinematic vertical drama series in hours instead of weeks?

That vision is Shine — the world’s first autonomous multi-agent AI video studio engineered specifically for serialized vertical micro-dramas.

What it does

Shine transforms an idea into a fully-produced, multi-episode vertical drama series through an intelligent multi-agent pipeline and an interactive WebGL non-linear editor (NLE):

Autonomous Multi-Agent Scripting Suite: A network of 5 specialized agents (Director, Story Skeleton, Adaptation Strategy, Script Agent, and Supervision Agent) collaborate to generate a complete 20–50 episode narrative arc with 3-second hook pacing, character dialogue, and cliffhangers.
Persona Studio & 98.4% Character Consistency: Anchors characters using up to 8 facial reference slots and LoRA models, ensuring character likeness, hairstyle, and wardrobe consistency across dozens of episodes and varying camera angles.
Interactive 9:16 WebGL Timeline Editor: A full in-browser NLE powered by OpenVideo (@openvideo/core, @openvideo/engine-pixi, and @openvideo/timeline). Creators can preview 60 FPS WebGL scenes with zero cloud wait, split/trim clips, layer tracks (VIDEO 1, AUDIO 1, SUBS), and use keyboard hotkeys (Ctrl+Z, Ctrl+Y).
Neural Voice & Multi-Speaker Dubbing: Multi-speaker TTS with emotion tags (Angry, Whispering, Sarcastic), pitch/speed modulation, and microsecond-level auto-alignment between audio duration delta ((\Delta t)) and video cuts.
Viral Trend Radar & Retention Engine: Multi-region trend scanner (US, Southeast Asia/VN, China, Latin America, Japan/Korea, Europe) that analyzes trending tropes, hashtag velocity, and scores scripts with a first-3-second retention prediction algorithm.
Smart Multi-Platform Distribution: Automated AI cover generation, viral hashtag optimization, and 1-click publishing directly to TikTok, Instagram Reels, and YouTube Shorts.

How we built it

We built Shine with a modular, enterprise-grade architecture spanning front-end rendering, back-end orchestration, and specialized AI skill pipelines:

AI Multi-Agent Intelligence: Powered by Google Gemini 3 models via @google/genai (Google GenAI SDK) and Vertex AI. We implemented a 3-layer architecture (Decision → Execution → Supervision) with structured JSON outputs and modular skill files (script_skeleton.md, script_scene.md, production_storyboard.md, trend_radar.md, compliance_check.md).
Real-Time Video Editing Engine: Built on OpenVideo (@openvideo/core, @openvideo/engine-pixi, @openvideo/timeline), utilizing PIXI.js WebGL canvas rendering for 9:16 vertical mobile viewports (1080x1920 @ 30 FPS), custom GLSL transitions, and keyframe animations.
Frontend Studio App: Developed in Vue 3 + Vite + TypeScript, using Pinia for centralized state management, TailwindCSS and Element Plus for a responsive dark-theme studio interface, and Vue I18n supporting 6 languages (en, vi, zh, jp, es, fr).
Backend & Streaming: Built on Node.js & Express, SQLite (better-sqlite3) for local episode caching, MongoDB for series metadata, and Socket.io for real-time streaming of agent reasoning states and async rendering progress.
Observability & Testing: Automated testing with Playwright Chromium E2E, custom mock data linters (check-quality.js), and strict 6-locale i18n parity validation (check-i18n.js).

Challenges we ran into

Cross-Shot Character Consistency: Standard diffusion and video models alter facial traits between shots. We tackled this by building an anchor extraction pipeline in the Persona Studio that feeds multi-angle facial reference images into generation prompts to lock down facial meshes.
Sub-Millisecond Multi-Track Audio-Video Alignment: Translating dialogue into different languages changes speech length ((\Delta t)), which normally breaks scene pacing. We engineered an auto-conforming algorithm in @openvideo/core that automatically extends or trims B-roll clips based on TTS waveform duration.
Bridging WebGL PIXI.js Canvas with Reactive Vue State: Coordinating the imperative WebGL rendering stage with Vue 3's reactive Pinia store required careful synchronization of playhead cursors, resize observers, and command-pattern action stacks.
Agent Hallucination & Script Quality Control: Large language models often forget cliffhangers or character relationships over a 30-episode series. We solved this with the Supervision Agent, which audits generated scripts against series-level story bibles before writing scenes to the database.

Accomplishments that we're proud of

100% Real Engine Integration (Zero Mockups): Successfully integrated a live WebGL canvas, multi-track OpenVideo timeline, and real Gemini API connections into a cohesive studio workspace.
Autonomous Multi-Agent Fleet: Built an end-to-end pipeline where 5 distinct AI agents collaborate to plan, write, check, and storyboard entire serialized micro-dramas.
Full 9:16 Mobile-First Video NLE: Implemented a non-linear editor tailored specifically for vertical cinematic storytelling, complete with dynamic subtitles, audio ducking, and GLSL visual transitions.
Comprehensive Quality & Localization: Achieved 100% test passing across 16+ live API test suites, automated Playwright real-browser verification, and complete internationalization across 6 global languages.

What we learned

Agent Specialization Beats Monolithic Prompting: Breaking script production into specialized roles (Skeleton Architect, Adaptation Writer, Quality Gate Supervisor) yields dramatically better narrative tension than asking a single LLM to generate an entire series.
Creators Need In-Browser Control, Not Just Black Boxes: Full-AI generation is magical, but creators demand the ability to split a clip, adjust dub timing, or swap an asset on a timeline. Combining autonomous agents with a real NLE is the ultimate creative multiplier.
Structured Data Modeling is Key to Serialized Drama: Modeling Series, Episodes, Scenes, Characters, and Audio Tracks as relational entities enabled seamless cross-episode memory and asset re-use.

What's next for Shine - AI Micro-Drama Video Studio

Voice-Driven Conversational AI Co-Director: An interactive director in the editor where creators can say: "Make the background rain heavier and punch in for an emotional close-up on Mara at 00:04."
Neural Lipsync & Video Synthesis Integration: Integrating models like Google Veo and LivePortrait directly into the headless rendering pipeline for photorealistic lip synchronization across all localized dubs.
Creator Marketplace & Interactive Paywalls: Expanding the platform to let creators monetize customized LoRA character packs, buy/sell virtual set environments, and publish interactive choose-your-own-adventure short dramas.