Our goal is to make shopping more comfortable. Searching for products can be very time consuming and getting detailed information about a product you consider buying is also not the easiest. There are already apps out there that try to assist with these tasks, but the implementation isn't really user-friendly (Finding and scanning the barcode of the product as an example)
What is SmartShopper
SmartShopper is an app for android smartphones that provides an AI based product scanning.
SmartShopper is designed to be easily usable, while still giving important information to the user.
Creating shopping lists is a piece of cake with SmartShoppers intuitive implementation of one.
With SmartShopper you can take a picture of a product and the app will display all important information about it.
With SmartShopper you can also take a picture of an entire shelf and the app will highlight the products that you want.
How to use SmartShopper
Shopping with SmartShopper consists of two/three phases.
In Phase 1 you create your shopping list; Just like you would normally, but on your phone in an great UI. You can add products to your list by just specifiing a keyword and a quantity. When you are done with the shopping list you can switch to Phase 2
Phase 2 is the shopping and scanning phase. You can take pictures of products or shelfs and the app will display relevant information. If you scan a single product you can see the price and other useful information and also have the option to add it to the cart, so it will be checked on your shopping list If you scan a shelf with multiple products the (not checked) products from your shopping list will be highlighted.
Phase 3 is only a hypothetical phase that is not (yet) implemented. Phase 3 will be the phase where the customer can actually speed up the purchase of the products through the app.
How it is made
SmartShopper consists of a backend and a frontend.
The frontend ist programmed with Ionic, which is a popular cross-platform mobile application framework.
The backend is programmed in Python 3 using the Tensorflow and Keras Frameworks. It is powered by a convolutional neural network.
Client and Server are communication via an API that is made with the Flask framework. Rough concept:
- Client sends picture to server
- Server sends it through the CNN
- Server evaluates the response of the CNN and transforms it to a JSON response for the client
- Server sends the response to the client
- Client evaluates the response
- Client displays relevant information
Challenges we ran into
One of the biggest challenges was obviously creating the CNN to detect a single product and multiple products. Another challenge was the efficient communication between the client and the server. And the third challenge was to reduce the concept (while still retaining the core ideas), so we would be able to create SmartShopper in under 48 hours.
What we learned
First of all, as half of our team is from non-german speaking countries and different cultures, we learned where the difference between developers from different countries are. Turns out there aren't any big differences. At least some of us also greatly improved our English-skills through that. We also learned how to organize a team-project and how to work in a team. And last but not least we got to now different fields of coding, like machine learning and app development.
It was great. Getting to know other students from other countries was a great experience for all of us. We also had much fun while developing SmartShopper.
But will SmartShopper emerge into in the future?
Our initial plans for SmartShopper were much bigger than what we implemented in the app. So at least some of us would like to continue developing SmartShopper, so it may become a real product in the future. The main goal for SmartShopper is gonna be "Simplify shopping as good as we can"