Inspiration
We were inspired by the stories of people with visual impairment, who struggle with getting a to get a good idea of their own looks and appearances. People with such impairment have to rely on others to confirm if their appearance is suitable for their outing. We read stories of people having to send selfies to friends and family to get an opinion on their appearance. These stories inspired us to develop Mirror, a digital talking mirror to let people with visual impairment gauge their appearance and also get holistic feedback on how to improve their looks.
What it does
Our talking mirror is a mobile app that scans the user's face to generate spatial maps of the facial contours and hairstyles along with detecting all micro and macro facial features such as Acne, Face Spots, Blemishes, etc. This information is then used for the following:
- Let the user know about any facial imperfections that can be treated along with the best course of action for treatment.
- Use the user’s facial shape to suggest hairstyles that will suit the user or improvements to their current hairstyle.
- Ask the user for the occasion (formal, casual, or ceremonial) and suggest what makeup to wear (in the future the app could assist the user in applying the makeup as well).
- Suggest the user suitable clothes and accessories based on the occasion, facial features, and hairstyles. All of the above will be communicated through a voice assistant inbuilt into the app which the user can talk to.
Functioning
Firstly, we have a computer vision-based detection engine that performs multiple functions. Its primary purpose is to detect and localize various facial imperfections, such as acne, face spots, and blemishes, with incredible precision. Furthermore, using 3D reconstruction techniques, it creates a spatial map of the user's face, accurately determining the shape of their face, facial features, and facial hair.
Secondly, our app features an AI expert, similar to the well-known chatGPT, that has been specifically trained on information collected from experts in the beauty industry. The AI expert takes the features detected by the detection engine as inputs and generates personalized suggestions for the user. These suggestions may range from treatments for skin imperfections to makeup advice based on the user's facial features, hairstyles based on the shape of their face, and even fashion advice, including clothes and accessories appropriate for a given occasion.
Lastly, our app features a voice assistant interface that allows users to communicate and utilize the various features of the app. The voice assistant is designed to converse with the user in natural language, similar to the existing voice assistants like Siri and Alexa.
Summarising, the Detection Engine scans and understands the various aspects of the user's face, the AI experts give suggestions based on the findings made by the Detection Engine and finally, these findings and suggestions are communicated back to the user using our Voice Assistant.
Log in or sign up for Devpost to join the conversation.