Few shot preferences
system, camera and raspberry pi
In the BC building at EPFL where the LauzHack hackathon is being held, there are a lot of events and talks. Those events often times also serve food. Our goal is to recognize when food about to be thrown away, due to nobody eating it. By alarming hungry (or poor) students in the vicinity, they can fill their stomach, while less food is being thrown away.
What it does
A 4K Logitech Camera, connected to a Raspberry Pi, records images every now and then from the space where food is served in question. The images are divided in many smaller images due to the high resolution, and possible localization of the food :). The client (pi) sends the images to the server. The server then fowards the patches to IBM Cloud, where they get classified. If enough food is detected, the server will contact the snips board and announce that food is available for the snips board owner.
With few shot learning we allow it to learn favorite (and possible exotic) items of every student. This way everyone can get notified when an item that he/she likes is available on the 'menu'.
How we built it
Created a patching algorithm, picture extractor, client server communication, snips.ai board interaction, IBM Cloud API calls for classification. Using IBM credits, and console.snips.ai to create intents.
Challenges we ran into
Configuring snips. Although the conversational agent with on-hardware processing snips.ai is a very exciting product, the documentation and depth of solutions on Stackoverflow is just not that good yet. OpenCV is hard to run on the pi.
Accomplishments that we're proud of
Blurring of people's faces on the client (Raspberry Pi), before passing sensitive images onto the cloud. Patching the images, for food source localization. Snips board that mentions a sort of menu.
What we learned
Learned a lot about the IOT MQTT interface. Few shot learning is cool, expecially to personalize preferences for specific objects (e.g. M-Budget food) in our case. Working for 24 hours with little sleep is hard.
What's next for Reducing Food Waste with Hungry Students
Enabling support for multiple snips boards, such that everyone can get a free food notifier. To reduce food waste as a community :)!