BeautiSense

Figma prototype
Screenshot of app demo

Inspiration

The lack of inclusive and adaptive solutions in the beauty industry that cater to the needs of visually impaired individuals inspired us to tackle the accessible beauty problem that many face. We hope to leverage advanced technology and a keen understanding of the unique challenges face by different users to break barriers and level the playing field in the world of beauty.

BeautiSense is an intuitive application that combines the power of facial recognition and artificial intelligence (AI)-driven analysis to provide users with real-time audio feedback and personalized guidance throughout the makeup application process. As the ultimate beauty companion, our app transcends barriers and unlocks the world of beauty for everyone.

What it does

BeautiSense addresses current issues in the beauty industry by providing a dedicated makeup application tool that offers real-time feedback, personalized guidance, and fosters independence for visually impaired individuals. The app uses facial recognition to identify facial features and leverages computer vision and image processing to analyze the live video feed from the user’s camera. It then provides real-time audio feedback to ensure a polished and balanced makeup look.

The seamless integration of real-time audio feedback, facial recognition, machine learning, an intuitive user interface, and personalization ensures a transformative and empowering experience for users.

How we built it

Use OpenCV and dlib for face recognition and detecting the 68 landmarks on the face
Use numpy to convert the coordinates from the landmarks into the actual number of the points that we want
Define a VideoCamera object and create a Flask app
Import the VideoCamera object into our React app

Challenges we ran into

We joined the hackathon late and was unable to fully implement all the ideas and features that we came up with.

Accomplishments that we're proud of

Brainstorming an idea that would help tackle the accessibility beauty problem, empowering not only visually impaired individuals but also those without visual impairments
Learning OpenCV and dlib for facial tracking
Implementing the webcam feature into a React component