Studio One AI

• 

• 

• 

• 

• 

Please note I had no time to make the video, and I ran off credits 😰

Inspiration Filmmaking has always been one of the most collaborative and complex creative processes — requiring writers, directors, cinematographers, editors, and dozens of specialized tools. We asked ourselves: What if a single conversation could replace an entire production pipeline?

When Google announced the Gemini Live API with real-time voice interaction, we saw the opportunity to build something that didn't exist yet — an AI Director you could simply talk to. Not a chatbot that spits out text, but a creative partner that listens to your vision, writes your screenplay, generates consistent characters, produces cinematic footage, and assembles it all into a finished movie. The Gemini ecosystem — spanning text, image, and video generation — made it possible to build a truly end-to-end creative pipeline for the first time.

We were inspired by the idea that the next generation of filmmakers shouldn't need to learn 10 different tools. They should just need a story to tell.

What it does Studio One AI is a browser-based AI movie studio that transforms a conversation into a finished film. Users open a project, talk to their AI Director, and watch their movie materialize through a guided 6-step pipeline:

Script — Write and structure your screenplay into chapters with AI assistance. The AI can generate, expand, or restructure your story based on voice direction. Characters — Define your cast with names, roles, and detailed descriptions. The AI generates consistent visual references for each character using Gemini Image Generation. Scenes — The AI automatically extracts scenes from your script, identifying locations, time of day, characters involved, and cinematographer-grade visual descriptions for each shot. Generate — Each scene is turned into cinematic video clips using Google's Veo 3.1/3.0/2.0 models. Users can choose between quality tiers (standard vs. fast) and monitor generation progress in real-time. Edit — (Coming soon) A timeline editor to arrange, trim, and layer your clips. Export — (Coming soon) Final movie render with custom resolution and format settings. The entire experience is designed to feel like working with a creative collaborator, not operating software.

How we built it We built Studio One AI as a modern, full-stack web application using Next.js 16 with the App Router pattern, React 19, and TypeScript.

Frontend architecture: The app follows a feature-based modular architecture. Each pipeline step (Script, Characters, Scenes, Generate, Edit, Export) is a self-contained feature module with its own components, hooks, services, types, and constants. We used Shadcn UI (41 components) for the design system, Tailwind CSS for styling, Zustand for client-side state, and TanStack Query for all server-state management with optimistic updates and cache invalidation.

AI integration: All AI operations run through Next.js API routes that call Google's GenAI SDK server-side. We integrated:

Gemini 2.0 for text generation (script writing, scene extraction with JSON schema enforcement) Gemini Image Generation for character visuals and scene storyboards Veo 3.1 / 3.0 / 2.0 for cinematic video clip production with async polling for long-running operations Backend: Firebase powers the entire backend — Firestore for real-time data (projects, scripts, chapters, scenes, characters, video clips), Firebase Auth for Google Sign-In, and Firebase Storage for all generated assets (images, videos). Every data operation uses the Firebase SDK directly — no REST layer.

Data flow: The pipeline is sequential but non-blocking. Users can write their script, then move to characters while scenes are being extracted by AI, then generate video clips while still refining earlier steps. All state is persisted in Firestore and synced in real-time through TanStack Query subscriptions.

Challenges we ran into Type-safe AI outputs: Getting Gemini to return structured JSON for scene extraction (with specific fields like visual_description, camera_angles, lighting_design) required careful schema design and validation. We had to iterate on the JSON schema enforcement to get consistent, usable results.

Video generation latency: Veo video generation is asynchronous and can take 30–120+ seconds per clip. We built a polling system with exponential backoff that updates Firestore in real-time, so users see progress without refreshing. Handling edge cases — failed generations, quota limits, model unavailability — required resilient error handling throughout.

Character visual consistency: Generating consistent character visuals across multiple scenes is inherently challenging with current image models. We experimented with different prompting strategies and model selection (prioritizing free-tier models while falling back to others) to give users the best results.

3-day timeline: Building an entire movie production pipeline — from script to video — in 3 days meant making hard architectural decisions fast. We chose Firebase for zero-config backend, Shadcn for instant UI components, and a strict feature-module pattern to keep the codebase manageable under time pressure.

Accomplishments that we're proud of End-to-end pipeline in 3 days — From an empty repo to a working AI movie studio with script writing, character generation, scene extraction, and video production — all integrated and deployed.

Real Veo integration — We're generating actual cinematic video clips from AI-written scene descriptions, not mock previews. Users can watch their movie scenes come to life in seconds.

Production-grade architecture — Despite the hackathon timeline, the codebase follows a strict feature-based modular architecture with proper TypeScript types, query/mutation hooks, service separation, and toast-driven error handling. It's built to scale.

Seamless AI-to-UI pipeline — The handoff between AI generation and UI is invisible to the user. The AI extracts scenes → Firestore updates → TanStack Query invalidates → the UI updates in real-time. No manual refresh, no loading spinners hiding broken state.

Multi-model video generation — Users can choose between Veo 3.1, 3.0, and 2.0 in both standard and fast variants, balancing quality against generation speed depending on their needs.

What we learned Gemini's structured output is powerful — JSON schema enforcement in Gemini text generation made it possible to build reliable AI-driven data pipelines (scene extraction, character parsing) without fragile post-processing.

Firebase + TanStack Query is a strong pairing — Using Firestore as the source of truth with TanStack Query for caching and invalidation gave us real-time updates with minimal boilerplate. The onSuccess → invalidateQueries pattern is remarkably clean.

Video generation is the future, but still early — Veo produces stunning results, but the async nature and generation times mean UX design around video AI requires careful thought — progress indicators, graceful failures, and letting users continue working while clips render.

Feature-based architecture pays off immediately — Even in a 3-day sprint, organizing by feature (not by file type) meant we could build the Characters pipeline without touching the Script code, and the Generate step without affecting Scenes.

Shadcn UI is a hackathon superpower — Having 41 pre-built, accessible, customizable components meant we spent time on AI integration and UX, not reinventing buttons and dialogs.

What's next for Studio One AI Timeline Editor — A drag-and-drop timeline editor for arranging, trimming, and layering generated clips into a finished sequence.

Export & Rendering — Server-side video stitching (FFmpeg on Cloud Run) to combine clips with transitions, audio tracks, and titles into a final movie file.

Gemini Live Director — The crown jewel: real-time voice conversation with the AI Director using the Gemini Live API. Talk naturally about your vision, and the Director updates the script, generates scenes, and produces clips — all from voice commands.

Audio & Music Generation — AI-generated soundtracks and sound effects that match the mood and pacing of each scene.

Collaborative Editing — Multi-user project support so creative teams can co-direct in real-time.

Template Library — Pre-built story templates (short film, commercial, music video) with pre-configured scene structures to jumpstart new projects.

Built With

• cloud-firestore
• firebase-authentication
• firebase-storage
• gemini-2.0
• gemini-image-generation
• google-genai-sdk
• next.js
• react
• shadcn-ui
• sonner
• tailwind-css
• tanstack-query
• typescript
• veo-2.0
• veo-3.0
• veo-3.1
• zustand