We were interested in a project where we can connect two thing that we like - machine learning and augmented reality. Luckily, we found out that Kaufland project fits our needs, so we decided to make a project both for customers and stuff of Kaufland.

What it does

Kaufmented reality make classification the product on the shelf and shows info about this product in AR. For customer it shows everything that is in Kaufland database (like price, calories, etc.) and for worker - where to find this product in storage. Also for both of them it will show is there any mismatch at product-barcode relationship.

How I built it

We use a data provided by Kaufland and also make and label our own dataset to provide this to mobileNet SSD network. After this we include this network to swift project via coreML.

Challenges I ran into

Labelling a lot of data, working with AR and choosing/collecting from the all ideas that we had for this project.

Accomplishments that I'm proud of

The whole project was quite hard to make, so our team proud of all of functionalities.

What I learned

Working with AR and Swift was kinda new for us. The main thing is to find out how to classify and show many classes at the same time

Built With

Share this project: