Inspiration
The inspiration for the "How Do I Look?" app came from the desire to provide users with a convenient and reliable way to get feedback on their outfits for different occasions. Many people struggle with fashion choices, especially when it comes to color coordination, cultural sensitivity, and seasonal suitability. We wanted to harness the power of AI and image analysis to offer personalized fashion advice, taking into account various factors that contribute to a well-put-together outfit.
What it does
Image Analysis: The app takes in an image of a person wearing a specific style of clothing.
Outfit Description: It uses an image caption generator model to generate a detailed description of the person's outfit, including colors, clothing items, and accessories.
User Input: Users provide information about the occasion for which they are dressed up. This input is crucial in evaluating the outfit.
AI Evaluation: ChatGPT, another AI model, analyzes the outfit description and user input to assess several criteria:
Color Coordination: Determines if the colors in the outfit work well together. Color Appropriateness: Checks if the chosen colors are suitable for the occasion and context. Cultural Sensitivity: Evaluates whether the outfit respects cultural norms and avoids potential insensitivity. Seasonal Suitability: Assesses if the outfit is appropriate for the current season. Accessories: Considers the presence and appropriateness of accessories. Overall Rating: Provides an overall rating, such as "consider changing outfit," "could be better," or "on point," along with a title, "overall rating."
How we built it
BLIP-2 (Image Caption Generator Model): The app uses a pre-trained image caption generator model to analyze the input image and generate a detailed outfit description.
User Interface: The user interacts with the app through a user-friendly interface, where they upload their image and provide details about the occasion.
ChatGPT Integration: ChatGPT is integrated into the app to provide fashion advice based on the outfit description and user input.
Criteria Evaluation: Custom algorithms and rules are applied to assess color coordination, color appropriateness, cultural sensitivity, seasonal suitability, and accessories. The overall rating is determined based on these evaluations.
Feedback Presentation: The app presents the user with a detailed assessment, including specific feedback on each criterion and an overall rating.
Challenges we ran into
Foundation models: Compute for running the models is something we struggled looking for initially.
Evaluation Criteria: Defining and implementing the evaluation criteria for outfits in a way that reflects real-world fashion standards was a non-trivial task.
Accomplishments that we're proud of
AI Integration: Successfully integrating AI models for image analysis and natural language processing to provide accurate and relevant fashion advice.
User Engagement: Creating an engaging and user-friendly platform that helps individuals make better fashion choices.
Feedback Quality: Ensuring that the feedback provided by the app is constructive and informative, helping users understand how to improve their outfits.
Real-world Applicability: Developing an app that can have a real impact on people's daily lives by boosting their confidence in their fashion choices.
What we learned
Image-to-Text Model architecture Fine-tuning Foundation models Using OpenAI API for using gpt3.5-turbo
What's next for How Do I look?
Real-time Image Analysis: Implement a system that can analyze images in real time. This could be particularly useful for event planners, social media platforms, or e-commerce sites looking to ensure the content shared aligns with their guidelines.
Image Enhancement: After assessing the appropriateness of an image, you can use a GAN to generate an enhanced version of the image that meets certain criteria. For instance, if an image is deemed appropriate for a party but the lighting is poor, the GAN could enhance the lighting and colors to make it more suitable for the context.
Virtual Wardrobe: Integrate a virtual wardrobe feature that suggests alternative outfits for images that may not be appropriate for a given context. Users can virtually try on different outfits generated by the AI
Log in or sign up for Devpost to join the conversation.