Inspiration
We wanted a conversational copilot that can run a full-funnel ad campaign, brief to strategy to creatives to predicted performance, without stitching together a dozen tools. The goal: live signals from YouTube/X, grounded TikTok insights, an opinionated multi-agent loop that produces a data-backed plan with end-to-end video generation and refinement.
What we learned
- Orchestrating Grok-powered agents with strict constraints and JSON validation.
- Pulling real signals from YouTube Data API v3 and X search, then compressing them into usable prompts.
- RAG vs MCP for offline vs live data: how to blend local doc recall (TikTok Creative Center) with real-time API signals.
- Balancing speed vs quality (model choice, prompt size, retry/timeout handling).
How we built it
- Conversational refinement: a chat with Grok turns free-form input into a structured campaign brief.
- Multi-agent orchestration: an orchestrator routes the brief through a Research agent (YouTube/X live signals + RAG), a Critic agent (constraints), and a Strategy agent (plan generation) until approval.
- Creative pipeline: Video ads are generated and refined with Open-Sora (used here as a stand-in for Grok Imagine).
- Performance prediction: creatives are embedded with ImageBind and scored by a Transformer-based model to predict engagement and rank ads before spend (dataset prepared; training pending).
Challenges
- Maintaining strict JSON schemas and validation rules while preserving output quality.
- Latency vs. quality trade-offs: balancing model choice, prompt size, retries, and timeouts to keep the system responsive.
- Normalizing imperfect real-time data (e.g. irrelevant tweets) without losing coverage.
- High computational cost and iteration latency make video editing loops expensive and still an active area of research.
- Evaluation of agentic pipelines: measuring correctness, stability, and convergence across multi-agent loops remains non-trivial and requires custom metrics.
Built With
- faiss
- framer-motion
- grok
- grok-4-1-fast-non-reasoning-for-agents-youtube-data-api-v3
- imagebind
- imagebind-(embeddings)
- next.js
- offline)
- open-sora
- python
- tailwindcss
- tiktokengagementcenter
- transformer-predictor-(training-pending;-x/tiktok-engagement-dataset-prepared)-next.js-(app-router)
- typescript
- x-api
- x-search-api-faiss-+-sentencetransformers-(`all-minilm-l6-v2`)-for-rag-open-sora-(video-gen
- youtube-data-api-v3
Log in or sign up for Devpost to join the conversation.