The lack of inclusive and adaptive solutions in the beauty industry that cater to the needs of visually impaired individuals inspired us to tackle the accessible beauty problem that many face. We hope to leverage advanced technology and a keen understanding of the unique challenges face by different users to break barriers and level the playing field in the world of beauty.

BeautiSense is an intuitive application that combines the power of facial recognition and artificial intelligence (AI)-driven analysis to provide users with real-time audio feedback and personalized guidance throughout the makeup application process. As the ultimate beauty companion, our app transcends barriers and unlocks the world of beauty for everyone.

What it does

BeautiSense addresses current issues in the beauty industry by providing a dedicated makeup application tool that offers real-time feedback, personalized guidance, and fosters independence for visually impaired individuals. The app uses facial recognition to identify facial features and leverages computer vision and image processing to analyze the live video feed from the user’s camera. It then provides real-time audio feedback to ensure a polished and balanced makeup look.

The seamless integration of real-time audio feedback, facial recognition, machine learning, an intuitive user interface, and personalization ensures a transformative and empowering experience for users.

How we built it

  • Use OpenCV and dlib for face recognition and detecting the 68 landmarks on the face
  • Use numpy to convert the coordinates from the landmarks into the actual number of the points that we want
  • Define a VideoCamera object and create a Flask app
  • Import the VideoCamera object into our React app

Challenges we ran into

We joined the hackathon late and was unable to fully implement all the ideas and features that we came up with.

Accomplishments that we're proud of

  • Brainstorming an idea that would help tackle the accessibility beauty problem, empowering not only visually impaired individuals but also those without visual impairments
  • Learning OpenCV and dlib for facial tracking
  • Implementing the webcam feature into a React component

What we learned

  • OpenCV
  • Flask
  • SpeechSynthesis
  • Challenges that visually impaired individuals face while applying makeup
  • Accessibility VoiceOver feature

What's next for BeautiSense

We hope to continue building the rest of the app and implement the key features.

Built With

Share this project: