MomentMatch AI

MomentMatch AI: The right ad, at the perfect moment.

Comment

Inspiration

I was inspired to build this project by the greatness of TwelveLabs, an incredible project standing alone as perhaps the best video understanding platform outside of Google but with a much better user experience. I was incredibly excited to employ it in a useful product and integrate the output with ChatGPT's web search tool in order to get realtime information to enhance the video context with product recommendations.

What it does

MomentMatch AI allows a user to upload a video which is subsequently Analyzed and Searched. The Analyzation returns Brand Mentions moments and Ad Opportunities, describing where a product was mentioned and where, based on video context, an ad would be a great fit. Each of these timestamped moments are associated with contextual product recommendations returned from the web. The Searched video returns clips in which highlights are features and then associated with products again. A Dashboard is in place for each video breaking down estimated engagement and CPM.

How we built it

The application was scaffolded and subsequently built with Cursor using the Claude 4.5 Thinking model. The technology stack consists of NextJS, React, Typescript, DrizzleORM, NeonDB, and Vercel hosting. It uses TwelveLabs for video analysis and ChatGPT for product web search.

Challenges we ran into

Cursor had trouble sometimes integrating TwelveLabs REST api seamlessly so I had to touch up JSON processing. TwelveLabs Search API did not provide any context about whats happening in the video so I had to spend time integrating the greatly powerful Analyze API which returned timestamps and video context for brand and ad moments.

Accomplishments that we're proud of

I'm incredibly proud of the seamless flow from upload to analysis and most proud of the ChatGPT integration which pulls live products based on context. A mention of AG1 Supplements will return a product recommendation and link for the item.

What we learned

While I was already familiar with TwelveLabs, I learned how to use the Search api to return formatted JSON and to intelligently integrate the output with ChatGPT's web search tool in order to retrieve ad products with context.

What's next for MomentMatch AI

Users, Authentication, a scrubbable timeline for the video with recommended ad text and images. Enhance search functionality with more categories and improve the Analysis prompt.

Built With

Share this project:

Updates