Inspiration
Every meal at USC Village ends the same way: a tray slides into the dish return and whatever is left on the plate disappears. Nobody records it. College campuses throw out an estimated 22 million pounds of food a year, and USC dining has almost no signal about which dishes come back uneaten.
USC Dining does ask. There are QR codes on every table that let students text feedback, but the associate director for residential dining told the Daily Trojan that not many students submit responses. The answers that do come in are broad 1–5 ratings ("food quality", "cleanliness") that never say which dish was the problem.
We wanted feedback that doesn't depend on anyone filling in a form. The plate already tells you what students thought of the food. We just had to read it.
What it does
Plateprint is a small camera above the dish return. For every plate that comes back, it:
Finds the plate and every leftover food on it, Measures how much of each food is left (in ml, then grams), Matches each leftover to a dish on that day's Village menu, and Logs the result to a staff dashboard that ranks dishes by returned weight, with filters by meal, station and date, plus a weekly PDF report and a committee email summary.
It stores plates, never people: frames are processed in memory and never written to disk, and any frame containing a face is dropped before it reaches the model.
How we built it
Data. We photographed 85 real plates at USC Village and split them 66 / 19 by plate, so photos of the same plate never appear in both training and validation. Three plate groups were held out entirely for testing.
Labels. Hand-drawing masks for 85 photos wasn't possible in 24 hours, so we built an autolabeler: Grounding DINO finds each food from its name, and SAM 2.1 turns each box into a mask. We then filtered the output (choosing the real plate, dropping bowls drawn over food, removing duplicate masks) and flagged any photo where a named food got no mask.
Segmentation. We fine-tuned YOLO26n-seg at 1024 px, adding 200 copy-paste composites to the training split to show it more food combinations. We exported it to ONNX so the rig runs without PyTorch. On the held-out plates:
Model Mask mAP50 Mask mAP50-95 Asparagus AP50 v1 (640 px) 0.755 0.545 0.379 v2 (1024 px) 0.756 0.615 0.623
Volume. Depth Anything 3 gives relative depth from a single photo. To make it metric we use the plate as a ruler: the plate's known diameter (21.59 cm) over its diameter in pixels gives the scale ( s ) in cm per pixel. We fit a plane through bare plate pixels, read each food pixel's height ( h_p ) above that plane, and integrate:
$$V \approx \sum_{p \in \text{food}} \max(0,\, h_p)\, s^2$$
Grams are then ( m = \rho \cdot V ), using a bulk density ( \rho ) for each food category.
Menu matching. SigLIP 2 compares each leftover to the names of dishes on today's menu in that food category only, so "poultry" can only become one of today's chicken dishes. Below a confidence or margin threshold it answers "unknown" rather than guessing.
Backend and dashboard. A FastAPI service with SQLite runs the pipeline and stores results, and a Next.js + TypeScript dashboard shows them to dining staff. The capture rig uses a motion trigger to grab one still, sharp frame per plate, and OpenCV's YuNet face detector as a guard that fails closed. The repo has 358 passing tests, with lint and type checks in CI.
Results on real plates. In our live end-to-end test on four real Village plates, every food was matched to the correct dish on that day's menu (6 of 6).
Challenges we ran into
The autolabeler kept missing food. Bowls were detected over the food inside them, and a single multi-food prompt confused Grounding DINO, which returned more labels than boxes. Running one pass per food name fixed both problems. Edge halos in the height map. Depth bleeds across food edges, so bare plate next to food looked raised and skewed the plate plane. Excluding a thin margin around each food (3% of the plate diameter) fixed it. A "robust" plane fit we tried looked better in tests but halved the volume on a real rice plate, so we left it off by default. Phantom regions. Tiny 0 ml slivers were being matched to dishes, so anything under 2 ml is now dropped. Environment fights. Depth Anything 3 pins old NumPy while OpenCV 5 needs NumPy 2. We lost real time to Colab resets and install conflicts. What we learned A known object in the frame can turn relative depth into real measurements. Splitting data by plate instead of by photo is the difference between an honest score and an inflated one. Privacy works best as part of the design, not added at the end: no frames on disk, a face guard that fails closed, and no student data at all. Limitations (being honest) Grams are not yet validated against a kitchen scale. We had no scale and the dining hall was closed. The camera's focal length is uncalibrated; our estimate differs from the photo metadata by 15–35%. The dataset is small (85 photos). Some dishes (rice, burger, short ribs, crepe) have no held-out evaluation yet, and stacked fries are only measured on their top layer. The same rice photo gives 137 ml in Colab and 119 ml on a Mac, a gap we haven't explained yet.
What's next for Plateprint
Weigh real plates to validate and calibrate the gram estimates. Calibrate the rig camera. Run a one-week pilot at USC Village with USC Hospitality and compare our numbers against their own waste audits. Expand the dataset to every station.
UGA cut plate waste by 19% in one semester with a plate-waste camera. We think USC can too, and that it can help USC on the way to its STARS Platinum goal for 2028.