A project for Dubhacks 2018 using OCR and Algolia to help people with dietary restrictions classify food that's good and bad for them based on a menu image file.

The pipeline is as follows:

  1. You go to a restaurant, but realizes that you have a few food restrictions you need to adhere to.
  2. There are tons of things on the menu and you don't know what you can and cannot get.
  3. You take a picture using
  4. parses the menu into list of dishes (In the future will use Algolia to support typos and synonyms, and ideally have a feature for users to add unique menu items as a keyword to its original dish name and improve search accuracy)
  5. searches the ingredients of said dish
  6. searches Algolia for the glycemic index/nutrition contents of each ingredient
  7. presents the menu to you in a whole new perspective!

Technical difficulties:

  • Limited OCR libraries for react-native
  • Many different glycemic index databases (difficult to standardize/choose a plan of attack)

Future plans:

  • Finish integrating OCR
  • Better UI/UX

Built With

Share this project: