Codinator

Inspiration

Ever struggle deciding what to wear from your closet on a given day? We believe wearing fashionable clothes can be correlated to increased self-esteem and better well-being, but thinking about fashion on a daily basis is too tedious. Why not build an app that recommends our fit based on a style we want? That's why we built the Codinator.

What it does

This is a web application that allows users to upload photos of their clothes to a virtual closet and, with the help of gemini AI, generates recommendations of clothing combinations based on a style the user desires. The website also allows users to try on the recommended clothes on themselves or on mannequin.

How we built it

We used react js for the frontend framework, supabase for the backend database and photo storage, and python's flask for designing an api linking the frontend and backend together. We also used Google's gemini, taking advantage of its multimodal capabilities to come up with the best AI based fashion recommendations based on photos of clothes we prepared.

Challenges we ran into

Trusting the recommendations that gemini produced was the biggest challenge. We got unsatisfactory images generated when we only fed images to the model, so we utilized gemini once more beforehand to generate secondary information about images such as their styles, description, occasion tags, primary and secondary colors and a formality score. By feeding secondary information along with the corresponding images, we were able to come up with a much more satisfactory outcome.

Accomplishments that we're proud of

We are proud of how our teammates were dedicated and excited to learn to use new tools and technologies. We have not used AI integration before, so throughout the process, we were challenged constantly during the process. However, we went through the process well as all of our teammates were determined to figure out the solutions.

What we learned

AI models are getting larger and larger and can handle complex tasks successfully. We were astonished by how Gemini can describe, generate, and modify given images. From this experience, we learned how important it is to understand the model and create better prompts, as this work can completely change the model's outcome.

What's next for Codinator?

Codinator is a closet of endless possibilities. We can implement methods where we can try on any clothes, whether that be from the internet, or text generated. If the cloth is from the internet, we can build a system where we can find the product with the lowest price and offer it to the user. With these improvements, Codinator would evolve from being just a fashion recommender to a comprehensive shopping assistant, empowering users to make informed decisions and virtually try on any clothing from around the world.