VISE-HACKATHON

About VISE (Visually Impaired Shopping Experience)

We have designed the app with visually impaired users in mind, and it consists of three primary features. Our main focus was to make it easier for them to use the app and buy products online independently and also support the after sales service.

E-Commerce feature

The first feature is the E-Commerce function, which enables users to purchase products using simple and consistent gestures throughout the app. With just three gestures, users can easily navigate the app without having to search for each button's location or remember them. The purchase process involves double-tapping and swiping on the screen, which provides vocal guidance to the user at each stage of the transaction. In the current iteration of the project, the purchasing is only allowed for selected Jo Malone London colognes and this feature can be easily expanded to different products and categories.

Product Identifier

The second feature is the Product Identifier. We understand that visually impaired users may face difficulties in identifying beauty products with similar packaging, especially under ELC. This feature helps users identify the product by capturing its image, and a scan is performed to process the text on the image and provide product details to the user through vocal guidance. This feature is mainly supported by Microsoft’s Azure computer vision service which allows the user to perform OCR(Optical Character Recognition) on the images taken by the user and produce the results.

Product Inventory

Usually, people rely on visual memory to recall the products they own but visually impaired users are unable to do so. This feature allows users to recall all the products they own after they have purchased. The feature is integrated with the E-Commerce feature where when a user purchases a product, the product is automatically added to their inventory. The user can then later refer to this inventory and check what products they own. A vocal guidance system is again present here to guide the user from the inventory. The products listed in the inventory can be discovered and the user can repurchase the products from that screen.

Inspiration

During our 12-weeks internship in the newly built KL Global Technology Center. We are deeply inspired by the VMA (Voice-Enabled Makeup Assistance). After some requirement gathering with the stake holders (Brands), we learnt that there is an opportunity to honor the brand name while giving better experience for the blind.

What it does

It simplify and give easier accessibility for the Blind and Visually Impaired user on their online shopping experience with the addition of the product identifier cool feature.

Accomplishments that we're proud of

With this project we were given a chance to present this to the whole Global Technology Centre around the world in hybrid. We are also given a chance to have a focus meeting to talk about our project with ELC current Inclusive Technology Lead, Lamia Drew.

What's next for VISE-HACKATHON

Our goal is to make shopping easier and more convenient for our consumers, especially the visually-impaired community.

Online reservation. Consumers will now be able to make a reservation to visit the store through our app with just a few clicks.
Loyalty program. that is eligible for both online and offline purchase. Every purchase earns points that can be redeemed for exclusive rewards and discounts. Just like the loyalty program the some brand implemented.
Handy product usage tracker. consumers can keep track of their product usage easily and make sure that they never run out of supplies. Our idea for this is to calculate the number of pumps used by consumers. We would estimate the amount of product used per pump and design a formula to calculate the total usage. The estimated usage would then be reflected on the app itself.
Store locator feature helps consumers to find the nearest store and get directions all from the convenience of your phone.
Switching to Azure database. It is the preferred platform by the company.Currently, our app is using Supabase which is an open source Firebase alternative to store our app data securely in the cloud.
Flexible billing options. make it easy to pay for the products you purchase. For our app, we will be adding more billing options such as ApplePay, GrabPay and more so consumers get to choose what works the best for them.
home concierge service. It works like a customer service where consumers can seek assistance from the beauty assistant through the app itself.

Besides all that , we always strive to keep improving our app and stay ahead of the competition. So below are the improvement we would like to add.

Our first improvement is on product identifier. We will improve the UI and UX of this feature to make the scanning process simpler. The second improvement is to increase the detail level of voice instruction for each feature. Consumer will be able to listen to step-by-step instructions on how to use each feature without having any knowledge in advance.
Next improvement would be allowing consumers to customise their experience with our app such as choosing their preferred language, background colour and more. This will make our app more catered to the consumers.
Finally, the last improvement is the addition of multi-language instructions. This feature will make our product accessible to a wider audience as consumers can choose the language they are most comfortable with which will greatly improve the user experience.

We provided some link to our Business Requirements Documents in confluence, Our Full Project Recordings and Slides.