Inspiration

We wanted a conversational copilot that can run a full-funnel ad campaign, brief to strategy to creatives to predicted performance, without stitching together a dozen tools. The goal: live signals from YouTube/X, grounded TikTok insights, an opinionated multi-agent loop that produces a data-backed plan with end-to-end video generation and refinement.

What we learned

  • Orchestrating Grok-powered agents with strict constraints and JSON validation.
  • Pulling real signals from YouTube Data API v3 and X search, then compressing them into usable prompts.
  • RAG vs MCP for offline vs live data: how to blend local doc recall (TikTok Creative Center) with real-time API signals.
  • Balancing speed vs quality (model choice, prompt size, retry/timeout handling).

How we built it

  • Conversational refinement: a chat with Grok turns free-form input into a structured campaign brief.
  • Multi-agent orchestration: an orchestrator routes the brief through a Research agent (YouTube/X live signals + RAG), a Critic agent (constraints), and a Strategy agent (plan generation) until approval.
  • Creative pipeline: Video ads are generated and refined with Open-Sora (used here as a stand-in for Grok Imagine).
  • Performance prediction: creatives are embedded with ImageBind and scored by a Transformer-based model to predict engagement and rank ads before spend (dataset prepared; training pending).

Challenges

  • Maintaining strict JSON schemas and validation rules while preserving output quality.
  • Latency vs. quality trade-offs: balancing model choice, prompt size, retries, and timeouts to keep the system responsive.
  • Normalizing imperfect real-time data (e.g. irrelevant tweets) without losing coverage.
  • High computational cost and iteration latency make video editing loops expensive and still an active area of research.
  • Evaluation of agentic pipelines: measuring correctness, stability, and convergence across multi-agent loops remains non-trivial and requires custom metrics.

Built With

  • faiss
  • framer-motion
  • grok
  • grok-4-1-fast-non-reasoning-for-agents-youtube-data-api-v3
  • imagebind
  • imagebind-(embeddings)
  • next.js
  • offline)
  • open-sora
  • python
  • tailwindcss
  • tiktokengagementcenter
  • transformer-predictor-(training-pending;-x/tiktok-engagement-dataset-prepared)-next.js-(app-router)
  • typescript
  • x-api
  • x-search-api-faiss-+-sentencetransformers-(`all-minilm-l6-v2`)-for-rag-open-sora-(video-gen
  • youtube-data-api-v3
Share this project:

Updates