VIRTUAL TRY-ON Project

Worried, out of thousands of online apparels available which fits you best?

Comment

Inspiration

Despite the convenience online fashion shopping provides, people are often concerned about how a particular fashion item in a product image would look on them when buying apparel online. ​ I seek to elevate this by creating a web application where a user can choose any garment they would like to purchase and then see a virtual model of themselves in that particular clothing, thereby judging for themselves if they should continue with the purchase.

What it does

Our idea focuses on the creation of a Virtual Fashion Try-On a system that seamlessly transfers the desired fashion apparel or item onto them.This definitely will help them purchasing apparel and accessories of their own choice and comfort avoiding the ​ process of ordering and returning unnecessary purchases. This ultimately will help them save time.

How I built it

This project is implementation of the paper VITON(here).

  1. Firstly, the user selects cloth and upload user's image on web-interface, which in turn is integrated with ML model at back-end.
  2. The model pre-process the image uploaded, then extract the parser (segmented-image) and detect the pose of the user-uploaded image using Open-Pose.
  3. These four things along-with cloth mask are feed as input to our first GAN (Generative Adversarial Network), the Geo-metric Matching Module (GMM) which returns the warped cloth.
  4. The warped cloth and everything used as input to first GAN is now transferred to second GAN, Try-On-Module(TOM) to receive the results.
  5. At last, to enhance the end-presentation of model to users, Image Super Resolution is used.
  6. Users can enjoy the final image which depicts the so chosen apparel being fitted to the user-uploaded image.

Challenges I ran into

  1. There were many incompatibility issues that I had to deal with while developing this project since many libraries, in use, keeps on getting updated.
  2. Being new to Google-Cloud Platform, it was tricky to host my Project.
  3. Automation. As mentioned earlier the open-source libraries take input from one folder and gives output to another. This issue took a lot take to get handled but finally it was done.

Accomplishments that I'm proud of

Final loss of generator on validation : 3.62001 Final loss of discriminator on validation : 0.003821

What I learned

  1. Creating a working model using opensource libraries.
  2. Hosting model on google Cloud Platform
  3. Grasp experience on working with tools like Pytorch, Open-Pose , torch-vision, tensorboardX and many others.

What's next for VIRTUAL TRY-ON Project

  1. Integrating the model with React.js
  2. While taking input from the users, which maintaining their privacy and security, I want to make sure the image does not hold any violent or nudity or any such content that violates law.
  3. Enhancing the final output is what I am working on right now.

Built With

Share this project:

Updates