Project Phoenix - Assistive Beauty E-Commerce

Skintone analysis
Product suggestions
Voice control

Inspiration

As I delved deeper into the realm of online shopping, I was struck by the lack of accessibility for people with disabilities. However, with the advent of AI-enabled user interfaces and user experience design, I was inspired to create a system that would leverage these intelligent systems to make online stores more accessible to everyone. My inspiration was fueled by various hackathon projects that I had the opportunity to study and learn from. Through extensive brainstorming and discussion with my peers, I was able to refine my ideas and come up with a unique approach to the problem of accessibility in e-commerce. I am thrilled by the prospect of bringing my innovative solution to life and making a positive impact on the lives of people with disabilities. By harnessing the power of AI, we can create a more inclusive and accessible digital world, where everyone has equal access to the benefits of online shopping.

What it does

Phoenix, an innovative assistive tool, aims to address this issue by leveraging cutting-edge technologies such as Automatic speech and wake word detection, Computer vision, and smart algorithms to provide users with a seamless and personalized e-commerce experience.

This sophisticated prototype features a mock Estee Lauder e-commerce website with Phoenix integrated, which detects users' skin tones and recommends products tailored to their needs. In addition, the tool assists users in navigating the website using only voice commands, allowing for an entirely hands-free experience. The try-on feature enables users to determine which products suit them best, while the share feature allows them to get a second opinion from their saved contacts.

By prioritizing the development of a sophisticated AI bot experience, Phoenix represents a significant step forward in advancing the state of the art in e-commerce accessibility. The platform's seamless integration into any e-commerce website makes it a powerful tool for enhancing inclusivity and providing a more personalized shopping experience for all users.

Although some features such as sign-up, sign-in, and multi-user support have been intentionally omitted in this prototype, this underscores the project's primary focus on exploring novel ways to leverage AI to enhance accessibility. In conclusion, Phoenix is a remarkable achievement in the field of e-commerce accessibility and holds great promise for creating more inclusive online environments for all users.

How we built it

Frontend: NodeJS and Svelte Backend: Flask and Python AI Models used:

Whisper: for speech to text
Web Speech API: for wake word detection and Text to Speech
OpenCV, Mediapipe: for facial detection and analysis

Challenges we ran into

Ideation: One of the biggest challenges in any project is finding a problem worth solving. In the case of this project, the challenge was to identify a problem in the e-commerce space that could be solved using AI and machine learning. The process of ideation required a lot of research and brainstorming to understand the pain points of users and identify gaps in the existing solutions. This involved analyzing user feedback, studying the market trends, and exploring various use cases to identify a problem that was both impactful and feasible to solve.
Deciding the architecture of the project: Once the problem had been identified, the next challenge was to decide on the architecture of the project. This involved determining the appropriate technology stack, selecting the right models and algorithms, and defining the overall system architecture.
Building the project: After finalizing the architecture, the next challenge was to build the project itself. This involved setting up the necessary models, integrating different components, and testing the project to ensure that it met the required specifications.

What we learned

I learned that makeup is not just about looking good. For many people, it's an important form of self-expression and a way to boost their confidence. Makeup can also help people express their individuality and create a unique look that represents their personality One of the key challenges in creating a makeup accessibility app was developing a skintone matching algorithm. I learned that skintone matching is crucial to helping people find the right makeup products that complement their skin tone. It involves a complex interplay of colours, undertones, and textures, and requires a sophisticated algorithm to get it right. I also learned that personalization is key to creating a successful makeup accessibility app. People have different skin types, tones, and preferences when it comes to makeup, and a one-size-fits-all approach simply won't work. By building a system that takes into account each user's unique characteristics, we can provide a more tailored and personalized experience. Through this project, I gained a deeper appreciation for the importance of inclusivity in the makeup industry. Many people with disabilities or different abilities face challenges in accessing makeup products, and our app aims to help bridge that gap by making the online shopping experience more accessible and inclusive Finally, I learned the power of technology to make a positive impact on people's lives. By leveraging AI and machine learning, we were able to develop a sophisticated makeup accessibility app that has the potential to transform the way people shop for makeup online.