Animatic

• 

  Script + photo upload, scene and storyboard style selection view

• 

  Agent trace view

• 

  Scenes reel play view

• 

  Scenes view with voiced dialogue and showing featured line

• 

  Expand a scene to see all its frames, with the option to regenerate or remove any one

• 

  References and descriptions for each scene

• 

  Pitch deck export

• 

  Last scene in the pitch deck

Inspiration

Concept art, a composer, and voice actors are expensive at the stage where a script is still just a script. So most scripts never get visualised at all, even the ones worth pitching. That gap was the brief for Agentic Cinema: The Summer Blockbuster Hackathon's Parallel track: build an agent that turns a screenplay excerpt into a visualized pitch package and show its work at every step rather than just handing back a finished file.

The moment that got me was seeing The Geometry of Rooms go from script to storyboards to score to voices to a reel.

How I built it

The backend is one Google ADK Agent, backed by Gemini, with nine pipeline steps registered as function tools: parse_script,wait_for_scene_selection, wait_for_casting,describe_character_reference,ground_visual_style, generate_storyboard, generate_score, generate_voice_lines,assemble_output.

The agent's instruction states the policy (parse once, let the user pick scenes, let the user cast characters, then for each scene: ground, board, score, voice, assemble), and the model decides what to call next based on what each tool returns.

Two of those steps are real human-in-the-loop pauses. The same orchestrator run blocks on an asyncio.Event until the frontend confirms scene selection or casting, instead of splitting the job into separate runs. If either times out, it falls back to a sensible default so the user is never stuck.

Storyboards use Gemini's native image model rather than Imagen 3, which returned a 404 and needed allowlisting we didn't have. The hackathon spec explicitly allows this substitution. Each scene is fed by a real Parallel Search call for a grounded cinematographic reference, plus a user-selectable rendering directive (realistic or animated) so the whole package keeps one consistent look instead of drifting from scene to scene.

Every tool call streams a structured trace event over SSE, so the frontend shows the agent's actual reasoning live: which beat it identified, which reference it grounded on, which frame just rendered.

For the frontend build and design inspiration, I used Lovable.

Challenges I ran into and learnings

Getting genuine agentic behavior to also feel fast and reliable turned out to be the real engineering work, well past the initial "make the pipeline run" milestone.

• A retry wrapper that quietly wasn't async. The Gemini call wrapper awaited a synchronous SDK call directly, so every image, TTS, and text call blocked the entire event loop for the full duration of the network request. No concurrency, no live trace updates, nothing. Fixing this with asyncio.to_thread and actually parallelizing a scene's independent image and voice calls was most of the latency win, not smarter batching.

• Two different flavors of Gemini failure. A 503 ("high demand") needs a wider backoff than the SDK's own couple of quick retries. A 429 (quota) needs something else entirely: read the server's suggested retryDelay and back off to that. On top of both, a concurrency cap keeps a burst of parallel calls from blowing through a free-tier per-minute quota the moment they all fire at once.

• Voice assignment that looked random. Voices were picked by hashing the character's name into a fixed pool with no gender awareness, so a female character could land on a male-leaning voice by chance. Worse, Python's built-in hash() is randomized per process, so the same character could get a different voice after every restart. The fix has two layers. First, the model infers each character's likely vocal presentation from the script. Then it picks within the matching pool using a stable hash, $\text{index} = \operatorname{SHA256}(\text{name}) \bmod n$, so the same name always lands on the same voice, not just within one process's lifetime.

• Consistency vs. privacy, genuinely in tension. A cast character's uploaded photo gets attached to every storyboard call for their scenes so the same face recurs. But that photo is deleted unconditionally the instant the run ends, which means regenerating a single frame afterward has no photo left to reference. The resolution was to anchor a regenerate to the scene's other surviving frames instead of the original photo. Not identical to starting fresh, but a principled way to stay consistent without quietly loosening the privacy guarantee.

• A crossfade that was secretly two cuts. The scene-to-scene transition looked like a hard jump despite a real opacity crossfade, because the outgoing scene was being freshly remounted for the fade. That reset it to its first frame's pan/zoom start position, regardless of where the scene had actually gotten to. The fix wasn't a longer fade. It was freezing a true snapshot of what was last on screen and only animating the incoming frame underneath it.

Built With

• asyncio
• docker
• fastapi
• gemini-tts
• google-agent-development-kit-(adk)
• google-cloud-run
• google-gemini-api
• google-genai
• parallel-search-api
• pillow
• pypdf
• python
• react
• reportlab
• server-sent-events
• tailwind-css
• tanstack-router
• tanstack-start
• typescript
• vite