Inspiration
Recently during a startup meetup in our city Bangalore, India, we got the opportunity to meet with people representing different companies ranging from startups to well established MNCs. One of the insights that we got from our discussions is that a lot of these companies/organizations are looking for solutions that can help market their products through an emotional connection with their audiences by making it personal to each and individual customers. They want to explore how GenAI can help achieve this objective but lack the time and resources to do so themselves. This problem along with our eagerness to build our skills in GenAI application domain motivated us to set out and build a tool for hyper-personalized ad creation.
What it does
Our tool, Marketing Genius, is a web app that uses agentic GenAI models to segment business customer data and generate customer specific hyper-personalized social media ads that business can generate through less than 2 clicks. The entire multimedia content is generated through Google's Gemini LLM model and covers channels like Instagram, Facebook, email and even generic YouTube Shorts.
How we built it
Our strategy was to use existing commercial models via 0 shot inference. We used Google's Gemini LLM model through Google Cloud Platform's (GCP) Vertex.AI API suite to integrate AI capability in our platform. The platform's frontend was built using Next.JS (in typescript) and backend was built using Python Flask. The frontend was deployed through Firebase hosting while the backend was deployed through GCP's App engine which also takes care about scalability.
We used 2 AI models - 1) Customer Segmentation - We used K-means algorithm to first cluster user profiles and then used LLM models to create summary descriptions for each cluster. This is done to display a simple description sentence on the frontend UI to enable intuitive selection of clusters by the end user. 2) Multimedia Content Generation - We used Gemini 1.5 Pro model to generate both text and image to create channel specific content using the information contained in a customer cluster.
The models were then integrated into the application and used to create backend API's for frontend to consume.
Challenges we ran into
1) Customer Data Segmentation We had to build a robust data cleaning and processing pipeline/agent that can intelligently consume different kinds and schemas of datasets. It was hard to create a Clustering algorithm recommendation system that selects the best algo to use for segmentation. The agent finds the relevant algorithms first by running each one on the dataset and comparing scores. We had to implement robust timeout measures and error handling because they would take indefinite time in case of large datasets or invalid data values.
2) Content Generation It was difficult to come up with the algorithm to create a custom video using GenAI. We ran into issues such as images not synchronizing with the narration and API's taking too much time to process the request. We overcame the challenges by parallelizing the process through asynchronous API calls and using image and audio buffers.
3)API Integration Accessing Facebook Ads creation APIs proved to be a challenge. In order to save costs, we tried using the sandbox account that Meta provides to developers to test out ad creation programmatically without incurring cost.
Accomplishments that we're proud of
1) We built a scalable system that can integrate independent AI agents working on different aspects of the system like a microservice. One agent keeps segmenting the customer data and pushes into a NoSQL database while the other agent pulls the data and creates Instagram, Facebook, email and YouTube ads.
Log in or sign up for Devpost to join the conversation.