VidCraft: Vids on your phone, uploaded. AI sees, spins tale.

Upload your clips, and Gemini AI instantly understands what’s inside—then transforms them into a short story - you choose the genre. AI then weaves those clips into one video and ElevenLabs narrates.

Comment

Inspiration

Every video holds a story—but most remain locked in clips on our phones. We wanted to turn everyday moments into cinematic narratives effortlessly. By combining visual intelligence with storytelling, VidCraft lets anyone become a storyteller in seconds.

What it does

VidCraft takes videos you upload, detects scenes and emotions using AI, and AI generates a short written story that a creator can edit. ElevenLabs then voices the narrative with lifelike speech, AI enhances or creates appropriate visuals, and delivers a shareable mini-movie - polished or allows creator to keep it raw and real (but adds the missing pieces in between raw clips).

How we built it

Google Cloud for fast, scalable video analysis and AI model hosting. fal for scene and object recognition to understand clips in context. Freepik assets to enrich generated stories with visual elements. Dreamina to transform text prompts into stunning visual art that matches the story’s tone. CapCut for seamless video editing and dynamic storytelling sequences. ElevenLabs for hyper-realistic voice narration.

Challenges we ran into

We are not the ones, who trained the AI, so results are constrained by off-the-shelf results of Gemini, ChatGPT, Claude, Llama, etc. Here is an example: https://gemini.google.com/share/832b13c89995 The results are not good "off-the-shelf." A production-grade move using AI requires a lot of human editing, so we preserved the flaws of existing AI. Syncing AI-generated storylines with video scenes in real time. Keeping narration and visuals emotionally aligned. Managing performance and latency at scale with multiple AI models working together.

Accomplishments that we're proud of

Building a fully automated storytelling pipeline in record time. Achieving high-accuracy scene detection and matching narrative tone. Creating a smooth end-to-end user experience from phone upload to finished story.

What we learned

How to orchestrate multiple AI models for a single creative output. The importance of emotional tone in storytelling automation. How sponsor tools can amplify creative workflows when smartly integrated.

What's next for VidCraft: Vids on your phone, uploaded. AI sees, spins tale.

Launching a mobile-first platform where anyone can upload clips and get a crafted story in minutes. Adding more customization: voice styles, visual themes, and story genres. Exploring real-time story generation for livestreams and events.

Built With

Share this project:

Updates