One of our team members regularly purchases canned soup, and she ran out without realizing it. She wondered if she could create something to automatically order more food when she ran out.

What it does

Diffy uses computer vision and OpenCV and RoboRealm to identify visual changes, to alert users when something is no longer there. Our demo involves identifying when things are missing from a pantry, so that new food can be automatically ordered.

How we built it

We first constructed a demo shelf, stocked with standard food items, and set up a live webcam feed. Then, we used OpenCV and RoboRealm to separate the output of the webcam into distinct frames and analyze sequential frames for visual changes, correcting for minute changes that we don't care about as well as changes in lighting. We then used Microsoft's Cognitive Services API for image recognition, the result of which is sent to a text-to-speech package to be read aloud. Finally, we built an Arduino with an ultrasonic distance sensor, so that the camera's output is only used and analyzed when motion is detected (the Arduino also helps the system avoid detecting a change when a person is between the camera and the shelf.

Challenges we ran into

Separating the live webcam video feed into discrete images; Putting the various independent components together; Setting up the picture such that the Microsoft Cognitive Services API can recognize it

Accomplishments that we're proud of

Getting the project to run successfully within 24 hours

What we learned

How to think in terms of processing an image in CV

What's next for Diffy

Porting Diffy from a computer into an online server, such that a low-power WiFi-connected camera can be set up anywhere


Logistics and item tracking; Detecting theft; Medical cabinets; Restocking supplies; Food notification at events (like HackHarvard!)

Share this project: