Consume Wise

Imagine going into any shop & being able to use your phone to find the most important details such as health and costing of an item you are about to purchase using the power of AR/VR! The future is :D

Comment

Inspiration

When shopping, we often found that we do not know every aspect of products we purchase. We often have limited information regarding health, value and economics when purchasing. Technologies such as Augmented Reality and Virtual Reality has been seen as a fun gimmick. We aim to show the utility in these technologies and to bring these technologies to light in a way which would help people. We would love to see these technologies as mainstream. We aim to connect people using technology, we strive to improve every aspect of our daily lives, to guide people to wiser choices, where simple navigation during shopping can be seen as only the tip of the iceberg.

What it does

Our system aims to give the shoppers all the information needed when purchasing a product. The system would recommend alternative products to shoppers, give a health rating of a product as well as guides shoppers in terms of the economic value of any potential purchases. The health rating and economic value are data points which we calculate in order to simplify the complexities associated with nutrition and value for money. The health rating is derived off of the nutritional information available on the product. Information such as calories, salt content and fat percentages are all used to determine this value and it is expressed in an easy digestible manner. The economic value compares the price to the price of similar products and determines how well it compares to the competitors. This in turn enables our users to make quick and informed purchasing decisions. The ConsumeWise platform consists of a AR and VR system. The AR system which can be used in a store - indicates the health rating, economic value, price and alternatives to these products. Our VR component - which can be used anywhere integrates with the AR system and uses products the user scanned to build a personalised online shopping experience for the user. The VR system displays the products users are interested in and enables them to make the same wise purchasing decision from the comfort of their home.

How we built it

We built this using Facebook Spark AR to give users real time information regarding a product. Spark AR scans the product and then displays the relevant information using Augmented Reality. The system is built with the goal to provide shoppers an AR shopping experience which uses the latest in cloud and back-end technology to deliver dynamic and valuable information. Our VR platform is built using Facebook React 360 and provides our shoppers with the ability to enjoy the same Consume Wise experience from their home. The React 360 application integrates with the information provided to ensure that user's interests are readily available. This can be demoed by following this link: https://consumewise.tech/. Our initial version of the AR application had issues with user experience and can be viewed from our initial demonstration video (https://youtu.be/uS7FutpSgL8). With the second version of ConsimeWise, the AR application was overhauled to improve the user experience.

Challenges we ran into

When testing the system, we found that branding and localisation of products can cause the system to not work optimally. We would love to eventually address this issue by introducing a more localised system. Another challenge was to integrate multiple item tracking into a single Spark AR filter. Although Spark AR has great item tracking capabilities, this limitation was difficult to work around. While building Consume Wise VR we faced the challenge of learning React as well as the React 360 platform. After many hours of tinkering and toying, we managed to create the Consume Wise VR platform. Which we are super proud of!

Accomplishments that we're proud of

We are proud of how responsive the system is and how it has the potential to become part of our daily lives. We are proud of the fact that we can help people make healthier and informed purchasing choices. We are proud that the system modernizes and enhances a persons shopping experience. We are proud of the fact that we could learn and create a system using Spark AR as well as React 360. We are proud of the fact that we could make a companion VR application to the AR system which integrates with the customers needs and wants. We feel like this was an accomplishment worth being proud of and we are fortunate to have had the opportunity to play with some of the latest and greatest tech Facebook has made available.

What we learned

During the development process, we learned that there is so many things in our daily lives we can do better and that anything is possible with technology. This community challenge / hackathon has given us exposure to some of Facebook's latest and greatest tecg. It was an amazing opportunity and a great learning experience.

What's next for Consume Wise

Hopefully we can take the system from a prototype to a fully fledged system which is being used around the world. We would love to use Spark AR to import the entire catalog of items from various stores. We would love to see the system usable across the entire world and with Facebook and platforms such as Spark AR we can almost see happening already. Lastly we would love for the ConsumeWise system to be production ready and to one day be part of a persons daily routine.

Built With

Share this project:

Updates