We couldn’t decide which frozen pizza to buy. Like us, many consumers are faced with a plethora of brand options on the shelves of grocery stores, but have no information regarding the brands and often resort to gut feeling to pick a product. However, wanting to end this constant uninformed decision-making, we created ConsumAR.

What it does

With American households throwing away over 16% of their annual food purchased, the wasted food in America totals to over $3 trillion dollars, and food is just the start. ConsumAR is made to create powerful ways to guide spending decisions by allowing users to instantly view ratings and reviews on the specific product and making an informed and effective decision. Additionally, next to the product, there will be a buy feature in which users can purchase the cheapest price of the product through Amazon, and in a week or so, depending on the product’s life cycle, the user will be asked to rate the product he/she bought. From a corporate side, our crowdsourced platform, firms can determine how their products’ ratings change as they make minor and constant modifications to their product. From both the corporate and consumer side, there are significant incentives that will encourage active participation in our platform. ConsumAR is driven by the goal to have no consumer make an uninformed decision and the intention that consumers will reduce the amount of products they waste.

How we built it

When entering a grocery store, the first step when using our application is to hold up your phone to a desired item. Built on a python program on Tensorflow, our Image Recognition functionality distinguishes any object after training the object within a pre-made neural network. Confidence Intervals are outputted and the object in the picture is recognized.

In our standalone AR interface, users can click on a button in a dialog window (containing rating of product), which pops up next to the object that has been detected. This adds the product to the user’s cart, and is stored in their purchased order. This AR functionality was built using React Native and Viro, a platform that integrates non-gaming AR applications efficiently.

Once a person has successfully bought a product, they can give a review for it. Then, this process repeats when other users go to stores and use our AR-based application.

Challenges we ran into

We had limited experience with Augmented Reality, so it was difficult to integrate it with different back-end applications like react-native. The UI was also very difficult to implement.

Accomplishments that we're proud of

Our Tensorflow-based image recognition software is extremely accurate, as the Neural Network ensures that accuracy always increases. In addition, we successfully created a standalone AR interface. Creating an integrated Augmented Reality and Tensorflow application has rarely been done, so we plan to improve this functionality in the future.

On the backend, a MongoDB database to store user purchases was created with a successful schema.

What's next for ConsumAR

In the future, we hope to be able to get more accurate product recognition for items beyond food and build a micro review sharing network between the user’s friends to create group recommendations. We would continuously improve the recognition capabilities, and streamline the integration between Tensorflow and Augmented Reality.

Built With

Share this project: