SayCut

SayCut — Make one's directorial dream come true. Speak a story idea and AI writes the script, generates images, and films video clips automatically.

Comment

What it does

SayCut is a voice-first agentic AI movie director. Speak a story idea — "a lonely robot finds a flower on Mars" — and the app autonomously writes a scene-by-scene script, generates keyframe images, synthesizes narration audio, and films cinematic video clips. Hit play and watch your movie back-to-back with subtitles, transitions, and a cinematic UI.

How we built it

An agentic loop on the server (Claude via Bedrock Converse API with tool_use) orchestrates the entire pipeline: generate_script → generate_image + generate_speech (in parallel per scene) → generate_video. The agent decides what to call and when — no rigid DAG, just an LLM with tools.

Four models, one pipeline:

  • Claude Sonnet 4.6 — orchestrator agent + script writer
  • Stable Diffusion 3.5 Large — keyframe image generation
  • Luma Ray v2 — cinematic video from keyframes (async via S3)
  • ElevenLabs TTS voice narration (via DigitalOcean)

The frontend is Next.js with Zustand (persisted to IndexedDB), Framer Motion for cinematic transitions, and SSE streaming so scenes appear in real-time as the agent works.

Inspiration

Filmmaking is one of the most collaborative art forms — writing, cinematography, sound design, editing — each requiring different skills and tools. We asked: what if a single voice command could orchestrate all of those roles? The multi-model hackathon was the perfect excuse. Each model excels at one craft (writing, imaging, filming), and an agentic orchestrator plays the director, calling each specialist at the right moment. The result is a creative tool where the barrier to entry is literally just speaking.

Challenges

  • Async video generation: Luma Ray writes output to S3 asynchronously, so we had to poll for completion and download the result — all within the agent's tool loop with a 5-minute timeout.
  • Base64 context bloat: Images and videos as data URIs would blow up the LLM context. We strip them from tool results before feeding back to the agent (summarizeForLLM).
  • AnimatePresence vs autoplay: Sequential scene playback broke because Framer Motion's exit animation delayed element mounting past when useEffect fired — solved with ref callbacks.

Built With

Share this project:

Updates