Carpo

Inspiration

8 years ago, I built an app for my gaming community. I had over 30,000 Tekken players using it, and I promised them all I'd make gifs for every attack from every character. I didn't realize it at the time, but that required making over 5,000 gifs.

How did I pull it off? I built a clipping tool to produce them. With a team of volunteers, we were able to tackle the project in a few months.

I decided to revive that tool but this time I wanted to enable an agent to work with me. Making clips is a long tedious process and requires a lot of precision. From finding the right moments, properly timing the clips and adjusting it to the way you need, it can be really taxing mentally. WebMCP lets an agent help with those tedious parts, while still allowing you to jump in and edit things by hand.

Sunil from Cloudflare talked about working with AI as having “your hands in the goop” alongside your agents. That's exactly how I feel about how we should be working with agents as well. It should feel like me and my agent crafting together. This experience is what I hope to capture with Carpo!

What it does

Carpo assists you in making clips of your videos of moments you might want to share on socials for marketing. It uses Cloudflare infra for storing and sharing the urls after they're made. Your agent can navigate it with WebMCP or you can use Project Think in the same way if you're not connecting via WebMCP.

How we built it

Built it iteratively with Coex, kept PRs small and used tons of architecture reviews with Matt Pocock skills. I had it read the old code from 8 years ago so most things went directionally the right way considering that it had a foundation to work of off that was familiar to me. The frontend uses React, TypeScript ,Vite and StyleX, Cloudflare Workers serve the app and API, using DOs to coordinate the FFmpeg containers. R2 stores the videos and transcripts. D1 holds the application records and I'm using Cloudflare Access for auth.

One thing that was really important to me was making sure that this worked without AI, the way I used to do it. If someone doesn't have access to an agent to interact through WebMCP, or if Think wasn't available, you can use this just like I did 8 years ago. I also think human approval is really important with these types of things so it's enforced through Think deterministically and we guide your agents to ask before making clips.

I found testing this manually to be very time consuming, so I used a flue agent to look at PRs descriptions, look at the diffs and test the features directly and provide screenshots as evidence. This made me feel confident that I can let it rip on some features and come back later.

Challenges we ran into

• GitHub outage. At first I was going to handle the browser testing through an agent through GitHub actions, but there was an outage that blocked that. This is when I looked into what Cloudflare has to offer and I learned about Browser Run and using Flue to navigate the site and test things I care about.

• YouTube. The original thing I built worked with YouTube and I do believe there is a path here where YouTube can work, but I was getting hit by all kinds of rate limits and stuff so I had to fall back from this. Very sad about it but it's something I want to explore further.

Accomplishments that we're proud of

Very proud that this is something that can be valuable whether AI is available or not. WebMCP just is there to give someone leverage and make them 10x more productive while they stay in the loop. I'm also really proud that it can be tested or used by browser agents as well. It can understand how to use the product through the structured tools!

What we learned

While we talk about "AGI" a lot and letting agents handle anything and everything, but this project was a good reminder that truly useful agents are going to need some sort of structure or well designed APIs to get anything done. I tried making videos and clips this way before, just asking an agent to find certain parts of videos where someone says a certain thing and more often than I would hope, it wouldn't do a great job. From what I've seen with Codex + Carpo, the success rate of getting exactly what I want is way higher now.

What's next for Carpo

I would love to push this further and make it easier and more efficient as far as sharing clips across different social platforms. As well as adding more editing abilities like sound boards or small visuals. I don't necessarily want Carpo to actual edit video by generating anything. I want people to be able to manage their own library of visuals and sounds and ask Codex to assist them in implementing it all. Maybe this could be a sick product some day!

Built With

• cfworkers
• cloudflare
• codex
• d1
• durableobjects
• flue
• gpt
• r2
• react
• sunil
• typescript
• video
• webmcp