Kroger Item FindAR

Inspiration

Our inspiration came from the idea that when going to the store, it can be difficult to find items that you are unfamiliar with. Suppose that you get coupons in the mail you would like to use on a new item, but do not know where it is in the store. New dietary restrictions could mean you do not know where to find the new food. To make this easier, and leverage new technology on smartphones, we decided to see if we could build an augmented reality experience which would highlight food items on the store shelves.

What it does

When a customer is in the aisle, there will be stickers on the ground that act as start points for the AR experience. When a customer scans one, it will be able to point out items on the shelves, taking into account dietary restrictions like gluten or peanut free options. Circles will show up circling shelf items that match the search criteria. It is also possible to look up a specific item and have the app show you exactly where that item is on the shelf, if it is in the area where the customer scanned the sticker.

How we built it

We built it through Android Studio utilizing Google's ARCore library for rendering the augmented content easier. There is a back end written with NodeJS which pulls item information from MongoDB. Towards the end of the hackathon we also wrote a front end in Angular to make it easier to add and remove items from the database.

Each item has an aisle and bin number and the stickers that are placed on the ground have a unique index. Using this information we can map the image index to a location in the store, and get the set of items back around that sticker.

Challenges we ran into

No one on our team had app development experience, so it was a learning experience for all of us. AR also poses challenges when rendering models in 3D space.

Accomplishments that we're proud of

We were able to get a working product by the end of the weekend, while not having prior android experience.

What we learned

We learned a lot of the concepts behind 3d rendering, and also a basic understanding of how to develop android apps.

What's next for Kroger AR Item Finder

Right now, the app assumes that the user is already in the bin for an item that they are looking for. In the future we would like to have full AR rendering of arrows which guide the customer directly to the item that they are looking for.

Built With

• android-studio
• angular.js
• arcore
• express.js
• java
• mongodb
• node.js
• sceneform