Monna

Snap a selfie, describe your dream look. Monna, the AI virtual makeup try-on, generates your perfect look and a personalized shopping list, so you know exactly what to buy to make it a reality.

Comment

Inspiration

Most makeup stores use sample books and testers to showcase their product catalog and provide customers with insights, sometimes even employing beauty consultants to enhance the customer experience. However, this approach can be confusing if a client is new to makeup or simply wants to try a new look without investing significant time. On the other hand, we see influencers across social media recording makeup tutorials and merely listing the cosmetics they've used. Both of these examples typically showcase only one look at a time, which can severely limit the choices a customer can explore.

To address this, we developed a virtual try-on service. This service allows companies to display their full catalog and offer customers customized ideas based on their individual needs. Furthermore, it assists retailers in planning their demand and reducing waste by minimizing the need for extensive stock and physical testers.

What it does

First, the user uploads a picture and prompts a look they wish to try. A customized multimodal model then suggests a makeup style based on the user's facial features and requirements. This suggestion is used to query products from a vector database and generate the virtual look on the user's image. Our goal is to provide robust cloud hosting and database management to support cosmetics and fashion retailers.

How we built it

This is a full-stack web application designed to perform computationally intensive tasks. It utilizes AWS Bedrock and ChromaDB, running on an AWS EC2 instance with a Gunicorn server behind an Nginx reverse proxy.

Our script powers a multimodal AI makeup try-on and product recommendation system built on a FastAPI application. We manage its lifecycle by establishing a connection to an external ChromaDB vector database via a lifespan context manager and initializing the AWS Bedrock runtime client for AI model calls.

The core endpoint executes a three-part workflow:

  1. Makeup Suggestion: It uses the Amazon nova-pro-v1:0 multimodal model to generate a suggested makeup style based on the input image and text prompt.

  2. Virtual Try-On: It feeds this suggestion and the original image to the Amazon nova-canvas-v1:0 model to create the "Generated Look" (the AI try-on image).

  3. Product Recommendation: Finally, it performs a vector similarity search within ChromaDB, using the Amazon titan-embed-text-v2:0 model, to find and list "Recommended Products."

The script also incorporates CORS middleware to permit cross-origin requests and includes utility endpoints for serving the front-end and populating the ChromaDB collection.

Challenges we ran into

We are two friends with backgrounds in international business management and computer science but had limited experience in full-stack development. This required us to learn all the aforementioned concepts from scratch. The most challenging part was successfully setting up all the used AWS services within a functional FastAPI application, given the multitude of choices and configurations the provider offers.

Accomplishments that we're proud of

Creating this solution in such a short time makes us feel proud and motivated to keep advancing the startup. While we deployed directly to production and still need to troubleshoot the system and implement other features we have in mind, we are at a stage where we can begin partnering with businesses and influencers.

What we learned

Beyond the technical knowledge acquired during this hackathon, we gained a better understanding of how web hosting and cloud service providers work from a business and marketing perspective. Even if some retailers have developed their own solutions, they have not yet fully explored Generative AI; their approaches are likely limited to traditional computer vision and image processing. This project is a strong potential use case for implementing AWS GenAI solutions within the cosmetics industry.

What's next for Monna

Monna is a startup in its first stage, and we are currently setting it up as a limited company in Sweden. This Proof of Concept serves as a demo to showcase the services Monna offers to potential partners across the beauty industry and beyond.

Built With

Share this project:

Updates