💡 Inspiration
As a solo developer, I face a brutal distribution reality: video converts 4× better than static copy, but producing high-converting 9:16 vertical video ads for TikTok, Instagram Reels, and YouTube Shorts is an agonizing bottleneck.
- The Blank Screen Fatigue: I used to waste hours inside timeline editors like Premiere or CapCut, agonizing over pattern interrupts, typography, kinetic safe zones, and audio alignment.
- The AI Video Latency Trap: Existing text-to-video AI tools forced me to sit through a 60–120 second blank loading screen before showing a single pixel. If the opening hook didn't land, that entire wait time was wasted.
- The Multimodal Token Explosion: When I looked at automating video quality evaluation, standard tools blindly dumped 1 frame per second into a vision LLM — devouring 20,000+ tokens and $0.05+ per critique without pinpointing why a viewer scrolls away in the first 3 seconds.
- The Post-Search Discovery Shift: Today's customers no longer just Google products — they ask Gemini, Perplexity, and SearchGPT. If my product's brand narrative and structured schema aren't cited by AI answer engines, my business is invisible.
I built Ashky to solve this entire problem end-to-end: an autonomous AI video ad director that generates high-retention video ads in seconds, inspects them with multimodal agentic vision, continuously evolves them through self-improving reflexion, and guarantees production reliability through Grafana Cloud MCP observability.
⚡ What Ashky Does
Ashky transforms a single 1-sentence product pitch into an optimized, broadcast-ready 9:16 vertical video ad campaign in a unified cinematic workspace:
1. Progressive "FirstFrame" Video Pipeline (~1.8s Idle)
Instead of forcing the user into a blank waiting screen, I decoupled video generation into a 3-scene narrative arc:
- Scene 1 (0–3s): High-contrast pattern interrupt hook. Delivered via Server-Sent Events (SSE) in ~1.8s (idle).
- Scene 2 (3–15s): Core mechanism & customer problem walkthrough (streamed in the background).
- Scene 3 (15–30s): Value anchor, proof, and clear call-to-action (synced into a master
.mp4).
This allows me to immediately evaluate visual pacing and typography while subsequent scenes assemble in the background.
2. Dual Generation Engine: Google Veo 3.1 + Turbo Kinetic Compositor
- Google Veo 3.1 Mode: High-fidelity vertical 9:16 cinematic video clips generated directly via Google Generative AI (
veo-3.1-generate-previewwith automated fallback toveo-3.1-lite-generate-preview). - Turbo Kinetic Mode: Ultra-fast deterministic compositor combining PIL procedural graphic generators, Microsoft Edge TTS neural speech synthesis, and multi-track FFmpeg kinetic typography in under 5 seconds.
3. Multimodal Vision Critic (Think-Act-Observe Loop)
Rather than indiscriminate 1-FPS frame dumping, I designed a Gemini 3.8 Flash Vision Critic using an agentic Think-Act-Observe (TAO) loop:
- Think: Formulates hypotheses regarding hook retention, typography contrast, and 3-second drop-off risks.
- Act: Selectively extracts 4 salient pixel keyframes via FFmpeg (
0.8spattern interrupt,2.2spacing transition,8.5sproduct mechanism,24.0sconversion CTA). - Observe: Inspects pixel contrast, subtitle safe-zone margins, and visual pacing.
- Measured Impact: Ingests only ~4,200–5,900 tokens per critique (vs. ~20,400 tokens baseline) — delivering a measured ~87–90% token reduction and 68.4% cost savings ($0.016 vs $0.052).
4. Closed-Loop Self-Improving Video Harness (Reflexion)
When the Vision Critic detects a defect (such as a visual pacing stall between 0.8s and 2.2s, caption safe-margin clipping, or predicted 3s drop-off > 40%), I can click "Evolve Video". The autonomous harness:
- Performs reflexion reasoning on the failure modes.
- Mutates prompt kinetic camera cues (injecting whip-zooms, volumetric lighting, or high-velocity tracking).
- Shifts caption safe zones inward by +40px.
- Re-renders an evolved
v2video, logging exact before/after retention scores and prompt diffs into a generational lineage ledger.
5. Generative Engine Optimization (GEO) & Schema Grounding
I built an intelligence arm that probes how AI answer engines (Gemini Search, Perplexity, SearchGPT) cite the product:
- Measures LLM Share of Voice (SOV %), sentiment, and narrative visibility gaps.
- Synthesizes search-optimized semantic headlines and target query matrices.
- Automatically generates machine-readable Schema.org JSON-LD structured data (
SoftwareApplication,VideoObject,FAQPage) ready for one-click copy and deployment.
6. Enterprise Observability via Grafana Cloud MCP & Autonomous SRE Copilot
I treated my AI creative pipeline with the rigor of a production microservice:
- Metrics: Native Prometheus
/metricsexporter tracking render latencies, token consumption, hook score distributions, and active campaigns. - Logs: High-velocity Loki event stream logging every scene render tick and agent decision.
- Grafana MCP Integration: Connects via JSON-RPC 2.0 to official Grafana Cloud MCP (
mcp.grafana.com), enabling bi-directional querying of telemetry. - Autonomous SRE Copilot (Sidecar): An in-app agentic assistant that inspects live pipeline traces, diagnoses latency bottlenecks, and provides actionable recommendations.
🛠️ How I Built It
- Frontend: React 19, Vite, TailwindCSS, Lucide-React, and native HTML5 Canvas particle visualizers. Built with dark-mode glassmorphic aesthetics inspired by professional non-linear video editors (NLEs).
- Backend Application Gateway: FastAPI on Python 3.11, structured around async endpoints, background worker tasks, and persistent Server-Sent Events (SSE).
- AI & Multimodal Reasoning:
gemini-3.8-flash: Powers the creative Director Agent, script formulation, and Multimodal Vision Critic.veo-3.1-generate-preview: Generates vertical 9:16 generative video reels with automated fallback toveo-3.1-lite-generate-preview.
- Media Engine: FFmpeg kinetic subtitle burning, safe-margin canvas computation, multi-track audio normalization, and Microsoft Edge TTS neural speech synthesis.
- Observability Stack: Custom Prometheus metrics registry (
prometheus_client), structured Loki log streaming, official Grafana MCP client (mcp.grafana.com), and pre-built Grafana dashboard configurations. - Infrastructure: Packaged as a Docker container and deployed globally to Google Cloud Run in
us-central1with--no-cpu-throttling,--cpu-boost, 2 vCPUs, and 2Gi RAM.
🧗 Challenges I Ran Into
Cloud Run CPU Throttling during Video Rendering:
- Problem: In initial deployments, long-running FFmpeg synthesis timed out after 120 seconds. Because FastAPI
BackgroundTasksexecute after the initial HTTP response, Cloud Run's default CPU allocation throttled CPU cycles to ~0 between requests. - Solution: Re-architected my Cloud Run configuration to run with
--no-cpu-throttling,--cpu-boost, and dedicated instance allocation (--min-instances 1), ensuring FFmpeg operations complete reliably in < 5 seconds.
- Problem: In initial deployments, long-running FFmpeg synthesis timed out after 120 seconds. Because FastAPI
The 20k Token Multimodal Trap:
- Problem: Traditional multimodal critique requires uploading the full MP4 or sampling 30 frames uniformly at 1 FPS, blowing past 20,400 tokens per evaluation and causing high latency.
- Solution: Created the Think-Act-Observe loop where the agent dynamically selects only 4 salient keyframe timestamps (
0.8s,2.2s,8.5s,24.0s), reducing token consumption to ~4,200–5,900 tokens (~87–90% reduction) while producing significantly sharper, second-by-second diagnosis.
Veo 3.1 Quota Resilience:
- Problem: Generative video models in preview tiers (
veo-3.1-fast-generate-preview) frequently hit regional 429 quota exhaustion. - Solution: Engineered a dual-tier model fallback strategy in
veo_engine.pythat defaults to standardveo-3.1-generate-preview, falls back toveo-3.1-lite-generate-preview, and provides the ultra-fast deterministic Turbo Compositor as an instant fail-safe.
- Problem: Generative video models in preview tiers (
Subtitle & Caption Sanitization in FFmpeg:
- Problem: Pitch text containing single quotes, apostrophes, or percentage signs (
%,') caused FFmpegdrawtextfilters to crash or drop captions silently. - Solution: Implemented automatic text file parameterization (
textfile=...) and safe-zone margin math, ensuring bulletproof rendering across all user inputs.
- Problem: Pitch text containing single quotes, apostrophes, or percentage signs (
🏆 Accomplishments That I'm Proud Of
- Sub-2s FirstFrame Delivery: Shipped a true progressive streaming pipeline where creators see Scene 1 rendered in ~1.8s (idle) instead of waiting 60+ seconds.
- Quantifiable Cost & Token Efficiency: Rigorously benchmarked and proved an ~87–90% reduction in tokens and a 68.4% cost savings using my Think-Act-Observe multimodal keyframe seeking approach.
- Closed-Loop Self-Improving Video: Implemented a working agentic reflexion loop where the system detects visual flaws, mutates generation prompts, re-renders the reel, and stores lineage diffs.
- Zero-Compromise Observability: Built full Grafana Cloud MCP and Prometheus integration into an AI creative application, treating LLM tokens, generation latencies, and retention scores with enterprise SRE rigor.
- Rock-Solid Production Deployment: Live on Google Cloud Run with 100% test coverage across foundation, agentic, video compositing, and MCP endpoints.
🧠 What I Learned
- Multimodal Efficiency Over Brute Force: In multimodal AI, how you feed visual data to an LLM matters more than dumping raw bits. Targeted keyframe seeking yields higher-quality reasoning at a tenth of the token cost.
- Agentic UX Requires Immediate Feedback: Generative AI tools feel magical only when they eliminate blank waiting states. Progressive SSE streaming fundamentally transforms how founders interact with video generation.
- MCP Unlocks True Agentic Observability: Integrating Model Context Protocol directly into the application enables LLM agents to become active system operators rather than passive chatbots.
🚀 What's Next for Ashky
- One-Click Multi-Platform Publishing: Direct OAuth integration with TikTok Creator API, Instagram Reels Graph API, and YouTube Shorts API for instant publishing and scheduled posting.
- Founder Voice Cloning: Integrating ElevenLabs / Google Audio to allow solo founders to clone their own voice for brand-consistent narrations.
- Live A/B Retention Optimization: Connecting live viewer retention analytics from TikTok/Reels back into the Ashky Reflexion Harness to automatically mutate future ad iterations based on real-world conversion data.
Log in or sign up for Devpost to join the conversation.