Scrap Saver

Universities waste ~3B lbs of food annually. Dining managers want sustainability, but data on waste is limited. Scrap Saver analyses leftovers to conserve money and the environment.

Comment

Inspiration

Coming from families where clearing your plate was a norm, stepping into UMich's dining halls was a shock. We saw hundreds of students scrape leave their untouched portions of chicken, rice, etc on their plates onto the dishwasher-conveyer belt. Across US college campuses, 4 billion pounds of food are wasted every single year. That represents 3.3 billion wasted meals, enough food to fill the Big House 10 times over and feed 3 million people for an entire year (2x the US homeless population). Food waste accounts for 10% of global human-caused greenhouse gas emissions and consumes 21% of agricultural water. We built Scrap Saver to rectify this.

What it does

There is a hardware component to this project. A Logitech Webcam takes a picture of leftovers, the Arduino UNO Q sends it to the laptop, which runs analyses using image segmentation and then updates the dashboard.

How we built it

We connected a Logitech webcam to an Arduino UNO Q in order to take pictures of plates. The Arduino UNO Q sends the images to our computer, which automatically uploads them to a Cloudflare R2 object database and Spacetime Database. Gemini is used to create bounding boxes and classify images, and Meta’s SAM 2.1 model is used to segment images. We calibrated the camera to factor in how much area each pixel takes up. Using the segmented images, we find the area of each wasted food, and then use data on environmental impact in order to find the total environmental impact of each food. The site is hosted on our laptop and deployed through a Cloudflare tunnel. Users have the option to input their menus and portions of each food served per day for the Gemini classifier and to calculate environmental impact per serving.

Challenges we ran into

In our first iteration of the product, we were using Gemini for both classification and segmentation. However, when we gave Gemini test data (half-eaten burgers from the internet), it would essentially guess and give different amounts of leftover food each time we ran the same photo. We then realized that Gemini is good at classifying which food is on a plate, but isn’t built to trace which pixels belong to it. We then researched segmentation models, and arrived on SAM 2.1 since it’s purpose-built for solely segmentation. Gemini now only identifies each food and draws bounded boxes around it, while SAM 2.1 traces the exact outline in each box.

We used East Quad dinner leftovers to test our Gemini 3.8 and SAM 2.1 setup. However, dirty trays can contain mixed arrangements of menu items from the same station, like vegetable stir fry and sticky rice. The standard bounded boxes were failing because they would group everything together. We noticed that the bounding boxes around the stir fry also covered the rice underneath it, so SAM couldn’t tell where which food ended and where the other began. To fix this, we had Gemini draw a separate box around each individual piece of food instead of one box per dish. This cut the share of food the model couldn’t identify by 39% (from 14.2% to 8.6%).

Accomplishments that we're proud of

We are proud that we were able to interview dining hall professionals from Bursley and East Quad. We learned that stations track leftovers by hand on printed sheets and that plate waste goes to compost without ever being weighed or documented. Staff also validated that there is a need for our product, especially from a labor and sustainability perspective and connected us with the right people for potentially testing our product in dining halls.

We are also proud of the real-world hardware that we built and integrated. Images taken on the webcam are automatically classified, segmented, and then uploaded to the dashboard with sustainability metrics.

What we learned

We learned through talking to dining hall staff that dining halls suffer from severe short-staffing, relying on handwritten paper sheets for kitchen leftovers while remaining completely blind to student tray waste. We also learned that there’s a very complicated reality for dining halls to measure demand on a daily basis. Attendance swings with weather, game days, exam seasons, and religious holidays. UMich kitchens make a six week dining forecast, so better waste data will improve their forecasts. We also learned that there’s no one-size-fits-all AI model. Gemini is great at recognizing food but falls short with tracing it, while SAM 2.1 is excellent at segmentation. In addition, we learned to test AI setups head-to-head on the same data and not to guess which configuration is better. Finally, this was our first hackathon, and we learned what it takes to go from an idea to a prototype in 24 hours. We started with customer discovery by walking into dining halls and talking to staff, and used our technical knowledge to develop a product. Our team brought together business and technical backgrounds, and we learned that each of the two depend on each other greatly.

Built With

Share this project:

Updates

Submission history