VisRecipe

Computer vision in recipe recommendation

Comment

Inspiration

We are poor students who are struggling to provide ourselves with nutritious meals. We want to help students like us by building an application which quickly suggests popular recipes to cook based on what's in their fridge. With the help of computer vision, we hope to relieve the inconvenience of having to manually find recipes.

User Persona

David is a student who does not like to cook. However, he lives on his own so he still have to cook sometimes. He doesn't like to go shopping for groceries so he would like to utilise whatever ingredients are available in his fridge. He is also a very hardworking student who wants to focus his time on his educational pursuit so he wants to be able to minimise the time spent on cooking.

What it does

Users take a few pictures of their fridge (although this function does not work properly yet). The system should correctly identifies what ingredients are available. Then it utilises pre-existing food recipe API to get a list of customised recipes. Users can instantly view the ingredients and instructions of the recipe by clicking on it.

How the prototype works

The instruction for darknet is in the file toRun. React website works by running npm start after npm install.

How we built it

After thorough discussion, our team decided to leverage an object detection system called YOLO t to o build the AI, and used Spoonacular API to get food data.

Challenges we ran into

We experienced several obstacles due to the technical difficulties of Computer Vision and integration of React web application with the machine learning neural network. We also ran into some complications with data scraping to extract the full instructions as a lot of APIs that we experimented with did not provide such services and each of websites recipes are linked to do not have similar structures. Although we did not manage to complete the prototype satisfactorily, we believe it can still be useful as a proof of concept.

Accomplishments that we are proud of

We managed to learn a lot about building a real application over the course of 24 hours. We are resilient and determined to make our product functional.

What we learned

We learned how hackathon is hosted.

Tech stack

Python Tensorflow React Node.js MDBootstrap Spoonacular API

What's next for VisRecipe

We love our idea and will continue to develop it further.

Link to repository

The Python code is not in this repo.

Built With

Share this project:

Updates