Inspiration

Imagine you've just been hired as the marketing head for the All-In Podcast, with one job: grow it to a million subscribers.

You'd need a whole team: editors to cut clips, someone to write and post, an analyst to read Instagram, someone running paid ads, and a lot of meetings to decide what to try next. Digital marketing leads and social channel heads spend most of their week coordinating agencies, editors, analysis and iterations, and what they learn rarely makes it back into the next edit.

AI tools today mostly act as copilots: they give you answers and analysis, then leave the work to you. We wanted something that actually does the work, end to end, and gets better every time it posts.

What it does

Growth Factory is an autonomous growth engine: a marketing factory of AI agents on your Mac. Its first production line is podcast video → Instagram Reels.

1. Tell it about your brand, once. Add your website, who the brand is for, its voice and boundaries. Agents read the site and draft your brand brief, build your style rules, write their own notes for the agents, and set up your goals and brief. You can edit all of it at any time. Connect your Instagram, and your factory is ready.

2. Drop in an episode. From one 10-minute All-In clip, the factory produced about 20 Instagram-ready Reels of ~30 seconds, with no editor involved. A swarm of agents listens, watches and judges every moment one by one, and the run ledger shows each step:

  • Whisper transcribes the speech
  • Agents watch every scene
  • Scouts propose moments
  • An editor cuts each clip: the punchline first, then the setup, word-by-word captions, a title card kept off the speaker's face, face-tracking vertical framing
  • A critic and a quality gate score everything against a rubric, explain why a clip is good or not, and rank them best first

3. Review, tweak, post. Change any clip in plain English, preview it as a Reel, and approve. Once approved, Growth Factory posts to Instagram for you.

4. Measure, learn, re-edit: the loop. This is the part current tools miss. The factory measures every clip over a 3-day window, compares it to your own baseline, and asks why. In our demo (simulated Instagram data):

  • Clips that opened on a question held 71% of viewers past 3 seconds
  • A clip that opened on setup held only 35%

The Strategist agent learns "question hooks hold viewers," updates the style rules, and the editor re-cuts the weak clip with a question-first hook, shown before and after. The next batch is made differently, and it keeps learning from every clip you put out.

5. It improves its own software. When the factory needs an editing capability it doesn't have yet, it writes a work order. A builder agent codes the extension, tests it, renders a real sample, and hands it to a human to review. The marketing factory contains a software factory.

How we built it

  • Mac app: React/TypeScript in pywebview, a local FastAPI engine, SQLite, and FFmpeg bundled inside the app
  • Listening and watching: MLX Whisper on Apple Silicon, scene detection and vision descriptions
  • Agents: scouts, editor, critic, Strategist and builder, each on local models (Ollama) or Claude through OpenRouter, chosen per role. Agents propose; code validates: timing snapped to sentences, the brand brief enforced (must-avoid topics are blocked in code), and every render checked
  • Craft engine: payoff-first teasers, title cards placed with Apple Vision face detection, emphasis captions, jump cuts, broadcast loudness, smart cover frames, and one-variable A/B variants
  • Distribution: Composio for Instagram (verified account, insights, publishing after approval), plus Supabase for sign-in and short-lived video hosting. Every post attempt is recorded before it happens, and an unclear outcome is checked, never blindly retried
  • Learning: measurement windows, a memory of past experiments, and a three-layer Strategist (craft → which moments to pick → campaign). Its confidence comes from deterministic math, so the model suggests and the numbers stay honest

Accomplishments that we're proud of

  • 10 minutes of All-In → about 20 ranked, captioned, ~30-second Reels in about 4 minutes, on a 16 GB MacBook. On Claude Haiku the agent work costs about 3 cents an episode
  • Hooks come from what was actually said: "You can't reverse engineer chips from wafers", "Why now? Why at all?". The agents found about 7 of 8 moments a human picked from the full transcript
  • The loop runs end to end on real clips: results → the Strategist learns a rule → the editor makes a real re-cut, before and after
  • The factory wrote its own feature: a progress-bar extension in 29 seconds for $0.05, with its 12 tests and a sample render passing, left for human review
  • Built honest: simulated data is labelled, a quality score is never mistaken for approval, and nothing posts without your sign-off

Challenges we ran into

  • One laptop, three heavy jobs: speech, vision and rendering compete for memory. We added memory checks, per-role cloud routing and one-click engine setup
  • Looking right vs. being right: a missing speech model once produced clips that looked fine but had no transcript. Now the engine refuses to run without speech and says why
  • Protecting meaning: shortening a hook can flip what someone said. Hooks must come from the transcript, and negations are kept
  • Shipping a real Mac app: a double-clicked app has no tool path (we bundled FFmpeg), packaging broke Keychain access (fixed), and live API replies differed from the docs (we parse what's actually returned)
  • Data isn't proof: organic metrics show correlation, not cause, so the Strategist reports confidence and evidence instead of promising growth

What we learned

Agents shine when each has a narrow job, a clear contract, and evidence a human can check. And the loop is everything: generating clips is the easy part, while learning from every post and changing the next edit is what turns a clipping tool into a growth engine.

What's next for Growth Factory

  • Close the first fully live loop: a real post, its measured 3 days, and the rule it produces
  • More channels: YouTube Shorts and TikTok
  • Paid growth: controlled A/B ad tests with budgets, where results can show cause, plus formats for D2C brands
  • A growing library of features the factory builds for itself, each approved by a human
  • Always-on cloud agents and a notarized Mac release

Built With

Share this project:

Updates

Submission history