Fridge to food

Inspiration

Being from Gen Z, we know the difficulties of cooking at home. Statistics show that only 40% of our generation does so. One of the clear barriers to cooking at home is having the confidence to cook good tasting, healthy meals. Many recipes (going beyond the humble sandwich) have been lost to the cookbooks. Our solution attempts to revive them: with a repository of over 10 000 simple recipes, we suggest our users food they actually want to eat. Giving this simple encouragement to cook at home goes a long way in creating healthier and happier citizens.

What it does

Fridge to food leverages the power of computer vision to recognize ingredients available in a fridge. Using a list of these items, we search a database of recipes to suggest the most convenient recipe that can be followed using the fridge items. These recipes are output on a display mounted to the fridge. To ensue the entire fridge can be viewed by the camera, it is mounted to a pulley that can move it vertically up and down to the different compartments of the fridge. This enhances the use of IoT in the system.

How we built it

The computer vision algorithm was built using OpenCV and the model trained using TensorFlow. The training was done using an assortment of online images from image search engines and from existing computer vision datasets. The list of recipes was obtained from an online dataset containing 13500 different recipes. The actuator mechanism that drives the pulley was controlled using Arduino, as was the LCD display that outputs the recipe.

Challenges we ran into

We have never done a hardware hackathon before so we were ill equipped with the know how for this project. For example, we did not know whether to laser cut, build or 3D print the model of our fridge. When we planned to 3D print the fridge and got a printing time of 3 days, we were forced to reconsider our assumptions about 3D printing.

Accomplishments that we're proud of

The solution we have built leverages the power of robotics in IoT, enabling better interaction of computer systems with the physical world. The camera can get a better view of items in the fridge because it can move: thus the recipe suggestions can be made more accurate. We feel this is significant because novel solutions such as ours need to be trusted to be made a viable business model: having high accuracy is one way of doing this.

What we learned

We have emerged more capable in building hardware prototypes, having understood that the art of improvisation is significantly more useful in a hardware hackathon. Our team has also gone from having different people specializing in ML and hardware programming to everyone having good exposure in the different fields.