The Fitting Room

Inspiration

Online shopping is hard. Sometimes you have a vibe or aesthetic in mind for clothes but need inspiration for items. Current shopping sites and search platforms can't directly find these items unless you input phrases like "white t shirt" or the specific piece of clothing you want. Moreover, you have to spend time combing through one brand website at a time. This difficulty in finding good clothing matches led to the thought: wouldn't it be nice to input a phrase or aesthetic and return clothing items that match?

What it does

The Fitting Room provides just that - a one stop shop that takes in a user input along with personalization like size, height, brands of interest, and price range and returns you items that match your input request. Say you were scrolling on Pinterest and had an aesthetic or event of interest in mind, on the Fitting Room, you could search "I want an academia aesthetic to wear to my class but it is cold outside" and be returned items of clothing related to that query based on LLM parsing and a similarity score. A user could then be directed to the website of each clothing item of interest. The Fitting Room also differentiates from sites like Pinterest with accessibility in regards to filtering by price and providing affordable clothing recommendations.

How we built it

We used web scraping to extract jacket, jeans, shorts, skirts, and tops data from Zara's online shopping platform, then loaded this data into a sqlite db. We then built Flask APIs and backend layers that a) interacted with ChatGPT4.0 to generate clothing item ideas based on the style, aesthetic, or general description inputted by the user, b) generated vector embeddings using openai-text-embedding-3-large model of all of the items in our db and stored them in a Pinecone index, c) conducted a similarity search between the vector embeddings of ChatGPT's recommended items and our items, and d) rendered the resulting items from our db (Zara's catalog) for the user to browse and navigate to through the Zara link.

Challenges we ran into

Our team is relatively new to web app development. One of our biggest challenges was integrating the front and back ends of our project for them to be able to communicate with each other. Additionally, we had originally planned to scrape clothing data from a selection of our favorite brands, but when we actually began developing the code to do so, we found that many sites had infrastructure in place to protect from scraping and we weren't able to scrape as much data as we had wanted.

Accomplishments that we're proud of

This is 3/4 of our team's first hackathon and we are proud of delivering a working project in such a short amount of time. We are particularly proud of the look of our front end design and effective use of LLMs and embeddings for clothing recommendations. We took time to identify an issue in our personal lives and built a product that solves the problem, which is something we are very proud and excited about.

What we learned

We learned how to work better with LLM integrations, and apply knowledge of embeddings that we had learned in our classes. We learned how to best utilize tools like Figma and Vercel to aid in front end design and development. We learned the different components of a web app and how they interconnect to build the interface the user sees.

What's next for The Fitting Room

The next steps for our project would be incorporating the top shopping brands' items into our service. We would want to use a hosted DB service so this project can be scaled to handle much more data. We also plan to incorporate another key feature into our service that would help users make use of the clothes, leaning into sustainability. How this feature would work is that the user would take a picture of an item in their closet that they already have and don't know how to wear/what to pair it with; we would generate multimodal embeddings so that we could find complementary items from our database to suggest to them, so they could find something similar that they already have/are interested in purchasing. The user would then have an outfit idea for something that they previously had trouble wearing, which would hopefully help reduce clothing waste.