Files uploaded to backblaze account
Input UI
Generated images and scenes
Generated video

StoryForge: AI-Powered Video Generation Platform

Inspiration

The explosion of AI video generation tools has made it possible to create stunning visual content, but the process remains fragmented and complex. Content creators, educators, and marketers spend hours piecing together scripts, generating images, animating scenes, adding voiceovers, and synchronizing music. We built Vido to unify this workflow—turning a single text prompt into a fully produced, cinematic video. Our vision was to democratize video creation, making professional-quality production accessible to everyone.

What it does

Vido transforms text prompts into narrated, scored, and captioned videos through an intelligent pipeline:

Smart Storyboarding: Google Gemini analyzes your prompt and generates a structured storyboard with scene descriptions, visual prompts, motion instructions, and narration scripts
Visual World-Building: GMICloud creates cinematic keyframes with consistent visual style across all scenes
Cinematic Animation: Kling technology brings keyframes to life with smooth motion and dynamic camera movements
Professional Audio: ElevenLabs TTS provides natural narration with customizable voices, while GMICloud Audio generates copyright-safe background music
Real-Time Experience: SSE streaming shows live progress updates as your video is built scene by scene
One-Click Export: Download finished MP4 videos with captions, ready for social media, presentations, or education

How we built it

Architecture Overview

Vido is a full-stack application with a pipeline-based architecture connecting multiple AI providers:

Backend (FastAPI + Python):

Genblaze Pipeline Framework: Orchestrates multi-stage AI workflows with built-in observability, retry logic, and concurrency control
Provider Abstraction: Clean interfaces for GMICloud (image/video/audio), Google Gemini (chat), and ElevenLabs/NVIDIA (TTS)
Storage Layer: Backblaze B2 for durable asset storage with automatic presigned URL generation
SSE Streaming: Real-time progress updates for engaging user experience
FFmpeg Composition: Professional video assembly with multi-track audio sync and caption rendering

Frontend (Next.js + TypeScript):

Custom React Hook: useVideoGeneration manages state, SSE events, and generation flow
Real-Time Updates: SSE event handling shows images/videos as they're generated
Modern UI: Tailwind CSS with glass-morphism design, Framer Motion animations, and responsive layout

Pipeline Stages in Detail

Stage A: Storyboard (Google Gemini)

Uses gemini-2.5-pro with custom prompt engineering for structured JSON output
Multiple JSON parsing fallbacks (direct parsing, markdown extraction, regex matching)
Storyboard persisted to Backblaze B2 with UUID-based keys

Stage B0: Reference Image (GMICloud Image)

seedream-5.0-lite model generates a 16:9 style reference image
Establishes visual consistency for all subsequent scenes
240-second timeout with automatic retry

Stage B1: Keyframe Images (GMICloud Image)

Parallel generation (max_concurrency=3) for performance
Each image inherits the reference style with scene-specific prompts
Presigned URLs generated for video stage input

Stage B2: Video + Audio (GMICloud + ElevenLabs)

Video: Kling-Image2Video-V2.1-Master converts keyframes to 5s/10s clips
Narration: ElevenLabs TTS (eleven_v3) generates voiceover per scene
Music: GMICloud Audio (minimax-music-2.5) creates instrumental background tracks
Duration Snapping: Automatically rounds scene durations to Kling-supported values

Stage C: Composition (FFmpeg)

Concatenates videos with crossfade transitions
Mixes narration and music with proper volume levels
Burns in captions with consistent styling
Generates final MP4 with H.264/AAC encoding

Provider Integration

GMICloud:

Image: seedream-5.0-lite (text-to-image, 16:9)
Video: Kling-Image2Video-V2.1-Master (image-to-video, 5s/10s clips)
Audio: minimax-music-2.5 (instrumental music)
Authentication: User-provided API key per request

Google Gemini:

Model: gemini-2.5-pro for storyboard generation
Temperature: 0.7 for balanced creativity
Response parsing with multiple fallback strategies

ElevenLabs:

Model: eleven_v3 with configurable voice ID
User-provided or server API key (optional)
Concurrent TTS requests for multi-scene generation

Infrastructure

Storage: Backblaze B2 with hierarchical keys (explainers/<run-id>/)
Containerization: Docker with Python 3.10+
Configuration: Pydantic Settings with .env support
Observability: Structured JSON logging with request IDs

Challenges we ran into

Provider Modality Conflicts: Using GMICloudVideoProvider for images caused modality errors. Fixed by separating providers: GMICloudImageProvider for images, GMICloudVideoProvider for videos.

Gemini Response Format: genblaze_google.chat() didn't support response_format. Implemented manual JSON parsing with fallback strategies (markdown extraction, regex matching).

Image-to-Video Input: Kling required HTTPS URLs for external_inputs, rejecting data URLs. Solution: Generate images first, store to B2, use presigned URLs.

Async Generator Limitations: Python async generators can't return values. Refactored to yield (sse_message, result) tuples with mutable result boxes.

Kling Duration Constraints: Kling only supports 5s/10s clips. Implemented snap_scene_durations() to round durations to nearest supported value.

FFmpeg Complexity: Synchronizing multiple video/audio tracks required precise timestamp calculations. Built robust composition with cumulative timestamp tracking.

API Key Security: Hardcoded keys were a security risk. Made GMICloud key user-provided per-request, ElevenLabs optional.

SSE Event Parsing: Events split across chunks caused parse errors. Implemented buffer-based parsing with line-by-line processing.

Accomplishments that we're proud of

Seamless Multi-Provider Integration: Successfully orchestrated five AI providers into a cohesive pipeline that transforms text to video with minimal input.

Real-Time Streaming Experience: Built responsive SSE system showing live progress with images and videos appearing as they're generated.

Professional Video Quality: Achieved cinematic output with consistent style, natural narration, fitting music, and accessible captions.

Robust Error Handling: Comprehensive error classification with retryable detection, user-friendly messages, and detailed logging.

Production-Ready Architecture: Modular components, environment-based config, type-safe APIs (Pydantic + TypeScript), and containerization.

User-Friendly Interface: Clean glass-morphism design, real-time progress indicators, one-click download, and responsive layout.

What we learned

Technical: Provider capabilities must be matched to modality; async patterns have constraints requiring creative solutions; FFmpeg requires careful timestamp management; SSE is simpler than WebSockets for server-to-client streaming.

Process: Start with a minimal working pipeline before adding complexity; log everything with request IDs for debugging; design for failure with retries and fallbacks; user experience transforms waiting into engagement.

Security: Never hardcode API keys; use per-request authentication; validate all external inputs (HTTPS URLs only).