Despite the convenience online fashion shopping provides, people are often concerned about how a particular fashion item in a product image would look on them when buying apparel online. I seek to elevate this by creating a web application where a user can choose any garment they would like to purchase and then see a virtual model of themselves in that particular clothing, thereby judging for themselves if they should continue with the purchase.
What it does
Our idea focuses on the creation of a Virtual Fashion Try-On a system that seamlessly transfers the desired fashion apparel or item onto them.This definitely will help them purchasing apparel and accessories of their own choice and comfort avoiding the process of ordering and returning unnecessary purchases. This ultimately will help them save time.
How I built it
This project is implementation of the paper VITON(here).
- Firstly, the user selects cloth and upload user's image on web-interface, which in turn is integrated with ML model at back-end.
- The model pre-process the image uploaded, then extract the parser (segmented-image) and detect the pose of the user-uploaded image using Open-Pose.
- These four things along-with cloth mask are feed as input to our first GAN (Generative Adversarial Network), the Geo-metric Matching Module (GMM) which returns the warped cloth.
- The warped cloth and everything used as input to first GAN is now transferred to second GAN, Try-On-Module(TOM) to receive the results.
- At last, to enhance the end-presentation of model to users, Image Super Resolution is used.
- Users can enjoy the final image which depicts the so chosen apparel being fitted to the user-uploaded image.
Challenges I ran into
- There were many incompatibility issues that I had to deal with while developing this project since many libraries, in use, keeps on getting updated.
- Being new to Google-Cloud Platform, it was tricky to host my Project.
- Automation. As mentioned earlier the open-source libraries take input from one folder and gives output to another. This issue took a lot take to get handled but finally it was done.
Accomplishments that I'm proud of
Final loss of generator on validation : 3.62001 Final loss of discriminator on validation : 0.003821
What I learned
- Creating a working model using opensource libraries.
- Hosting model on google Cloud Platform
- Grasp experience on working with tools like Pytorch, Open-Pose , torch-vision, tensorboardX and many others.
What's next for VIRTUAL TRY-ON Project
- Integrating the model with React.js
- While taking input from the users, which maintaining their privacy and security, I want to make sure the image does not hold any violent or nudity or any such content that violates law.
- Enhancing the final output is what I am working on right now.