Cutty AI

Inspiration

As someone who is constantly learning, building projects, and editing short-form content, I ran into a wall that almost every beginner editor faces: getting real, constructive feedback is almost impossible.

When you show a draft to your friends or family, they will usually say "looks great!" just to be nice. They don't know why a hook failed in the first 3 seconds, why the pacing dragged at second 6, or what sound effect would fix an awkward cut. I spent hours digging through random CapCut tutorials, Reddit threads, and editing packs trying to guess what my videos were missing.

I wanted a copilot that acts both as a senior creative director and as an honest editing buddy who isn’t afraid to tell you what sucks, while handing you the exact tools to fix it. That is how Cutty was born.

What it does

Cutty AI is an interactive short-form video retention auditor powered by multimodal AI:

• Multimodal Second-by-Second Audit: Instead of just transcribing audio, Cutty watches the video frames and listens to the audio track simultaneously to evaluate visual hooks, pacing, cuts, and audio balance.
• Synchronized Interactive Timeline: Pins key feedback moments directly onto the video player timeline so you can jump to the exact second where retention dropped.
• The "Bro Roast" Mode: An unfiltered, honest breakdown with real editor slang to tell you what feels amateurish and why.
• Instant Editing Asset Kit: Curates concrete, actionable assets tailored to your specific clip—including exact sound effects (SFX), bold typography pairings, transition effects, and hex color codes ready to copy into CapCut, Premiere Pro, or DaVinci Resolve.

How we built it

We designed Cutty as a high-performance, dark neo-brutalist workspace built for editors:

• Frontend: Built with React, Vite, and Tailwind CSS, featuring a responsive 9:16 player console, dynamic timeline scrubbing, and interactive mascot state feedback.
• Backend: Node.js with Express running containerized on Google Cloud Run.
• AI Core & Vision: Direct integration with Google's Gemini 3.5 Flash via GenAI. We pass raw video buffers to leverage native multimodal understanding (analyzing visual motion and acoustic dynamics in a single pass).

Challenges we ran into

The hardest part was definitely language and schema synchronization with the multimodal model. Video analysis returns complex temporal data, and getting the model to return strictly structured JSON with concrete asset recommendations—instead of generic placeholders—took extensive prompt engineering and backend validation layers.

On top of that, handling direct video payloads through HTTP proxies and base64 conversions while keeping the latency under a few seconds required fine-tuning our Cloud Run memory buffers and payload limits so the analysis felt immediate.

Accomplishments that we're proud of

• Achieving true multimodal video inspection that delivers actionable editing feedback in roughly 5 to 7 seconds.
• Creating a distinct personality for Cutty that bridges professional retention theory with approachable, mentor-like coaching.
• Building a full-stack production setup from scratch with Google Cloud Run and Vertex AI that actually solves a daily bottleneck for creators.

What we learned

We learned how far Gemini 3.5's native multimodal capabilities have come. Treating video as a first-class citizen rather than splitting it into detached audio transcripts and static snapshots opened up entirely new possibilities for creative analysis.

What's next for Cutty AI

• Exportable Timelines: Generating native .edl or CapCut/Premiere project files with markers automatically placed on the timeline.
• Audio Track Separation: Deeper audio analysis isolating voiceover, background music, and SFX levels.
• Batch Auditing: Auditing multiple draft variations at once to predict which hook will retain better before posting.

Built With

• express.js
• google-cloud
• google-cloud-run
• google-gemini
• javascript
• multimodal-ai
• node.js
• react
• tailwind
• vercel
• vertex
• vite