Decoding Food Items Using AI And Atrify-

Upload the single image of your grocery item and get all the information about that product or you can upload the image of the grocery shelf to get a whole bunch of information.

Comment

Inspiration

We are always curious about the food we buy and the food we eat. The most common questions we all have in our mind is, does this food healthy or not, does it contain preservatives or allergens, is this economic friendly, how much calories, fat, sugar, and salt in it. Most of us are not willing to read the entire scripts written on the food label. Thus we unknowingly harm us buy eating extra stuff creating a whole bunch of health problems. In the market, there are plenty of apps available to do the same thing. Examples: Healthy Food – Smart choices in the grocery store, Clueat –Food Indications, ShopWell – Healthy Diet & Grocery Food Scanner, Sift Food Labels. But most of these apps scan the bar code or use OCR(Optical Character Recognition) to get the nutrient information. The problem with such a system is that sometimes the bar codes can't be visible or very hard to find and scan it individually.

What it does

**1. Upload any grocery item , using deep learning and atrify api it will return with most needed nutrition information. No unwanted information, straight to the point

How we made it

Dataset https://github.com/PhilJd/freiburg_groceries_dataset

Total number of classes: 25

  1. BEANS
  2. CAKE
  3. CANDY
  4. CEREAL
  5. CHIPS
  6. CHOCOLATE
  7. COFFEE
  8. CORN
  9. FISH
  10. FLOUR
  11. HONEY
  12. JAM 13JUICE
  13. MILK
  14. NUTS
  15. OIL
  16. PASTA
  17. RICE
  18. SODA
  19. SPICES
  20. SUGAR
  21. TEA
  22. TOMATO_SAUCE
  23. VINEGAR
  24. WATER

To train the model use this link : Open In Colab

Run

To run use this, colab link: Open In Colab 1. First we train our model on efficient_net and made a tflite model file 2. Used it to make prediction on what is the grocery item is. ex: chocolate, soda etc. 3. Using the label name(chocolate,soda etc,) we made a query on brocade api to get gtin-14 number( https://www.brocade.io/api/items?query=$query). Intially we selected first and second gtin number 4. After getting the gtin-14 number we used atrify nutrient information api to get all the information about the product. Including fat, sugar, salt, allergence etc. 5. We built a Flask RESTful API web application to make things smooth 6. But we failed to identify the brand name. Thought about applying OCR. Tried pytesseract. But accuracy is too low and then we tried AWS textract. Even the state of the art OCR engine failed to identify certain text. Then we move on with manual entry of brand name

Challenges we ran into

1. Finding the dataset and training the model on Google colab taken much of our time 2. Developing a web also taken much of our time

Accomplishments that we're proud of

1. Built a deep learning and atrify api powered Flask application 2. Made a Tensorflow classificaion model for grocery item classification with **70% validation accuracy** **3. Combined multiple api with AI and computer vision to form a novel solution

What we learned

1. Building solutions on top of API **2. Tweaking hyper parameter or hyprer parameter fine tuning for accurate model.

What's next for Decoding Food Label Ingredients using AI and atrify

  1. We tried to use the Holoselecta data provided by autoidlabs,which include 109 classed for custom object detection training using TensorFlow. But the 2.1 GB dataset is too much for Google colab. We dropped the plan due to this limitation.
  2. Training on object detection-SSD_Mobilenet_quantised model will helps to deploy offline mode on mobile application, Fast inference speed and much more.
  3. Customer can just take the photo/video of the grocery shelf and all the information can be accessed in an instance in real-time

Built With

Share this project:

Updates