Smart Beauty Architecture

Beauty is a natural expression in the human being and when it is combined with different IT elements we can show our desire to help people in a more intelligent way.

Comment

Inspiration

Beauty products are designed and made to highlight the natural aspects and expressions of our bodies. From the shapes of the eyes, the size of the lips, and the contour of our face to showing the tone of our skin to represent our body in a more stylized way.

Therefore, my inspiration comes from the fact that always having high-quality beauty products available is of great help to people since many of them have the vitamins and nutrients necessary to have a better life. In addition, anyone has the right to have access to beauty products regardless of their physical condition, mobility, or any type of health condition.

What it does

Smart Beauty Architecture provides an innovative solution to provide a better shopping experience for all Estee Lauder customers. Through a system of User Interfaces, it can assist people with different special needs. For example, for deaf people, the architecture has an integrated camera that works through computer vision and Artificial Intelligence for the recognition of sign language made by the person in front of the camera. As a result, a human voice will be executed interpreting the sign language made, making known what you want to express to a hearing person in such a way that better assistance is provided when making a purchase or consultation.

In the case when a person in a wheelchair is physically present in the store, the Intelligent Beauty Architecture will have a screen located on the shelf. In such a way that the person will be able to select which product he/she wishes to buy, add it to the shopping cart, and make the payment respectively. Then, a mechanized arm will place the selected items in a bin at the average height of a person in a wheelchair so that he can take the item with his hands.

On the other hand, if the person has vision problems, the intelligent beauty architecture will give them the option to enlarge certain details of the product on the screen such as a zoom option on the screen to see the color of the item's tone, enlarge the size of the letters in relation to the description of the product on the label, an environment in 3D dimensions of the article to read its technical specifications.

The Smart Beauty Architecture will feature a graphical interface that is capable of being multi-language. In this way, if a person has a special disability but does not master the English language, for example, then the option of the article, menu, and description can be changed to at least the most used languages worldwide such as French, Spanish, Chinese, and Arab.

How we built it

Smart Beauty Architecture has a series of modules through a user interface. The user can interact with each module according to their need through the following actions:

• Tap-Screen. • Voice Recognition: using NPL for a bot. • Sign Language Recognition: built with a Convolutional Neural Network

Once the user selects the module and language of his choice, a series of options will appear on the screen. For example, a person in a wheelchair can select this image.

image

Then the software will ask you what type of article you are looking for according to a series of categories. For example skin care, foundation, lipstick, mascara, shadow, night cream, etc. Then, the user will be able to add the items to the shopping cart and finish the payment process using their credit/debit card. Then, a robotic arm will look for the items on the shelf levels, each shelf level has a series of sensors that register the position in which the item is located, and the arm by means of a gripper will take the item and take it to the available basket so that the person in a wheelchair can reach them.

image

Now if we assume that the user is deaf or hard of hearing, through the software you can select the hearing module.

image

Here, the hearing-impaired person can navigate through the application to determine which product they want to search for to purchase. If this person needs assistance from a hearing person, then they can select the option that says "Assistance". A representative will then approach you to provide advice. In the event that the representative does not master sign language by means of a camera and an integrated microphone, she will be able to speak. Then the message will be represented on the screen as sign language so that the deaf person can capture the message. Then, the deaf person can use sign language in front of the camera so that the signs are converted to a voice message through the use of cloud services and the hearing person understands what the deaf person wants to communicate.

The same would be the case if the person is mute and can only communicate through sign language. In this case, the user must select the following logo:

image

And then you can request assistance through the section of the following logo

image

Once the selection is made, a customer service person will be assisting the deaf person. At this point, the deaf person can make a sign using their language and it will become an output text for the hearing person. The hearing person can then use the microphone to send audio which will be translated into sign language for the deaf person on their side of the screen.

image

image

Dynamic Time Warping

And as an innovative factor in our proposal, we have decided to integrate Makeup filters that can be used through the camera of smart devices through the Instagram and Facebook applications thanks to the Spark AR software. The purpose is that all people can try the variety of products that Estee Lauder offers on their faces and help them make the decision about which products they want to buy and help them make their shopping experience much more fun and inclusive.

image

image

Challenges we ran into

In the case when a person is color blind, we have to work with the issue of contrasts with the colors that we want to present in the interface, in such a way that we can make available the options that we present in the user interface.

Another challenge will be to educate users with different disabilities so that they can adapt to this type of technology that is intended to be made available in stores.

Design and propose a solution that is diverse, equitable, inclusive, accessible, and fair for people with different disabilities.

Accomplishments that we're proud of

Due to our idea proposal, we can build a Smart Beauty Architecture by design that integrates different functionalities for all those people who have some type of disability. In addition, the proposed design allows for a new purchasing paradigm where the user is the main actor.

What we learned

People with disabilities have the same rights as everyone else. Technology can help us bridge the gap in terms of being more inclusive in different environments that surround us. We don't have to change people with disabilities, we just need to adjust or modify the environment so that they are part of what is around them. The solution proposal can be adjusted for different scenarios. Through the advancement of Artificial Intelligence, electronic components, software development and the services offered by the Internet, we can create an innovative and cutting-edge architecture that will help us change the traditional paradigm in stores. Providing the foundations for a new future.

What's next for Smart Beauty Architecture

Review User Interfaces / User Experience

Make a Neural Network prototype to test with OpenCV regarding the use of Sign Language

Feed the dataset with Sign Language.

Make a prototype for the recognition of Speech Recognition

Study different cases with people who have different disabilities.

Create a survey campaign aimed at people with disabilities to find out the degree of acceptance of our idea proposal. In addition, it would allow us to assess whether we are not including any relevant aspect for people with disabilities.

Built With

Share this project:

Updates