Makeup is not just a vanity project to make people look beautiful. Beyond the surface, there is an immense culture that fosters creativity, passion, self-confidence, and artistry. Unfortunately, although the makeup industry is a 500 billion dollar industry, matching foundation, which is the basis of most make up applications still relies on old ways of matching with trial and error. The hope through roboMUA (robo Make-Up Artist) is to utilize machine learning to make foundation matching easy, convenient and very inclusive.

What it does

  • Upload an image of the user's face to the platform,
  • Get the image classified based on 5 skin types,
  • Get a recommendation of which foundations that will match that particular skin tone.

NB: The recommendations include foundation names, company, product website, prices, images, and even a YouTube tutorial of influencers using that particular foundation to see how it suits that particular skin tone.

How I built it

  • Curated a dataset of over 2000 online by scraping images from the web with a python script and labeled the data into 5 skin tone types.
  • Trained the dataset with a Convolutional Neural Network image classification model with Pytorch on GPUs.
  • Deployed the ML model with Python Flask Web Application.
  • Hosted the application on Google Cloud.

Challenges I ran into

  • Curating the dataset for the model was difficult, mainly because of the interest in making sure dark skin was featured in the dataset and there was not a well-curated dataset that included dark skin folks.
  • Training the machine learning model with PyTorch. As a new beginner of PyTorch, it was difficult to find answers to some of the questions I had which could not be found even on the official PyTorch site and so had to resort to a lot of trial and error.
  • Deploying an end to end machine learning project that others could use was a huge challenge.

Accomplishments that I'm proud of

  • Improved my skills on deep learning and PyTorch significantly. I now have a better understanding of how deep learning models work.
  • Receiving feedback from dark skin users who were appreciative of their prominent inclusion in a makeup product.
  • I learned a lot about how product management and cloud technology works.

What I learned

  • Some times in technology, there are no straight pathways so you have to adapt and change methodologies in order to achieve results.
  • User Experience design is really hard.

What's next for roboMUA

  • Instagram filters to utilize AR to enhance the foundation match.
  • Add more skin tones to reflect the diversity of skin tones in reality.
  • Refine the model to get better accuracies
  • Add more functionalities
  • Improve UI design

Built With

Share this project: