Calories In A Picture

Use image processing to detect food objects (and more) and associate a calorie count with that item. Display that count along with the item on the screen.

Comment

Inspiration: My History class talks about food a lot and I was studying for that exam when I learned about Volhacks.

What it does: This program takes video from a Windows 10 based Computer's input source and runs a deep learning image processor over it. After finding the objects in the camera's field of view, it will display that object with the linked calorie count.

How I built it: It began by utilizing haar-cascade scripts but I soon found that method too failure-prone to be a usable model. After figuring out that it wasn't going to work with the cascade scripts, I decided to rewrite the program entirely. As I already have a little bit of knowledge with OpenCV, the process was not terrible in the slightest. I am now using a limited script to detect food objects using the CoCo dataset with a deep learning base.

Challenges I ran into: As stated before, I found out that haar-cascade scripts were not going to cut it for this project (They would but I didn't have the time to take 1000+ photos of each individual food object). After poking around a little more online, I found an awesome system for detecting objects and implemented that instead. Furthermore, the sheer exhaustion and energy-crashes repeatedly got to my head.

Accomplishments that I'm proud of: I am proud of the fact that I implemented this system properly and that it works exactly how I want it to.

What I learned: I learned a lot more about OpenCV and advanced python scripting than I would have ever hoped to learn before this. It was definitely different coming from a Java / C++ background but I would be more than happy to use this language more in the future.

What's next for "Calories By Picture": The only thing I didn't get to finish was the API implementation that would allow the program to query a food item and return a calorie value directly from the web. I would love to finish this and expand the amount of food items the program can connect.

Built With

Share this project:

Updates