Inspiration

Sourcing stock footage scene-by-scene after writing a script is a repetitive, momentum-killing challenge. That's when the idea hit: What if I could automate this?. ScriptToClips automates this process of finding, evaluating, and organizing stock media.

What it does

ScriptToClips bridges the gap between script writing and video editing by automating the most tedious part of video production: sourcing and organizing stock media. Here's the workflow:

Creators paste a script into ChatGPT using a built-in prompt to get structured scene JSON
They paste that JSON into ScriptToClips for validation
The app searches Pexels for matching videos or images per scene
OpenAI ranks results by visual relevance (with a heuristic fallback for cost optimization)
Creators preview results, replace weak matches, and select assets
Download everything as an organized ZIP with optional scene metadata

How I built it

Tech Stack: Express.js + TypeScript, EJS templates, Pexels API, OpenAI API, Archiver

Scene Validation (sceneJsonValidator.ts): Rejects malformed JSON upfront to save API calls.
Intelligent Media Pipeline (mediaPipeline.ts): Handles Pexels searches with concurrency caps and local caching.
Dual-Ranking System (clipRanker.ts): Combines OpenAI Vision scoring with a cost-effective rule-based fallback (resolution, aspect ratio).
Browser Session Persistence: Stores lightweight URL/ID references locally for stateless server efficiency.
ZIP Export (zipCreator.ts): Packs media and metadata into edit-ready archives.

Challenges I ran into

Balancing Automation & Control : Solved by surfacing top candidates and replacement options rather than forced auto-selection.
Managing API Costs : Capped ranked candidates (max 10), added heuristic ranking, implemented session caching, and made AI scoring optional.
Flaky APIs & Empty Results : Added 15s download timeouts, graceful feature degradation, clear error messages, and query editing.
Async Complexity : Managed via modular handlers, request-scoped state, and bounded concurrency.
Storage Limits : Persisted light references in the browser while rehydrating details on demand.

Accomplishments that I'm proud of

End-to-End Automation: Functional pipeline from raw script to downloadable media pack.
Resilient Dual-Ranking: Graceful fallback during AI rate limits or outages.
Human-in-the-Loop UX: Automated busywork without stripping creative agency.
Optimized Architecture: Type-safe across the stack, budget-conscious, and fault-tolerant.

What we learned

Solving Personal Pain Points: Living the bottleneck builds sharper tools.
API Orchestration Matters: Managing concurrency, quotas, and caching is as critical as core features.
Nuanced AI Vision: Context-aware asset ranking demands precise prompt engineering and solid fallbacks.
Upfront Validation: Strict input validation prevents cascading downstream errors.
Creator-Centric Design: Automation should eliminate tedium, not override human judgment.

What's next for ScriptToClips

Phase 1: Standalone Maturity: Native script parsing (removing external LLM steps), robust retries, and UI refinement.
Phase 2: Expansion: Database persistence, NLE integrations (Premiere, DaVinci), multi-provider media support, and user accounts.
Phase 3: Monetization: Tiered SaaS subscriptions, usage-based credit systems, and agency white-labeling.
Phase 4: Intelligence: Custom vision models, style-predictive suggestions, and team collaboration features.