Petugram: Smart Kitchen, Zero Waste

Inspiration

None of us set out to build a food app — we set out to actually use AI for something real instead of just reading about it. The idea came from a genuinely mundane place: staring into a fridge, not knowing what half the things in it were for anymore, and throwing out something that had quietly expired days ago. That happens in almost every kitchen, all the time. Once we started digging into it, we realized the real problem wasn't a lack of recipes online — it was that no recipe site knows what's actually sitting in your fridge right now. That gap felt like exactly the kind of problem AI is good at closing.

What it does

Petugram turns your fridge into the starting point for everything: what to cook, what to plan, and what to give away.

Digital Fridge — snap a photo, scan a receipt or barcode, or add manually; the app tracks expiry dates and warns you before things spoil. AI Match — an AI recipe engine reads your available ingredients and returns ranked recipe suggestions, complete with substitutes for anything you're missing. Meal Plan — auto-builds a 3–7 day plan around your fridge and preferences, then generates a grocery list of only what you're actually missing. Share — got extra food you won't use in time? Post it, let nearby users claim it, and coordinate pickup through an in-app inbox. Discover — find restaurants and recipes nearby with an interactive map. Impact — see real numbers: money saved, food waste avoided, CO₂ reduced — localized to your region. How we built it

We treated this as an AI-first build from day one rather than bolting ML onto a finished app afterward.

The AI layer does three separate jobs: a vision model reads photos and receipts to identify ingredients automatically, a T5-based recipe generation model proposes dishes from those ingredients, and — this was the part we learned the most from — sentence-transformer embeddings (MiniLM) re-rank those generated recipes by how well they actually match what's in the fridge, since raw generation alone kept suggesting dishes using ingredients we didn't have.

Around that AI core, we built a FastAPI backend that talks to MongoDB for persistence, and a vanilla JS frontend that stays lightweight on purpose. For location-based features, we integrated OpenStreetMap, Overpass, and Nominatim for restaurant discovery and geocoding, and Open Food Facts for barcode lookups.

Challenges we ran into

The hardest problem wasn't getting a model to generate recipes — it was getting it to generate useful ones. Early on, the model would confidently suggest dishes needing ingredients nowhere near our fridge input. Fixing that meant learning how to use embeddings for semantic similarity ranking rather than relying on the generation model's judgment alone — something none of us had hands-on experience with before this project. We also underestimated how much design work goes into a two-sided feature like food-sharing: claiming, notifying, and messaging between two strangers needed more state-tracking than we initially planned for.

Accomplishments that we're proud of

We're proud that this isn't a demo running on a hardcoded example — it's a live, deployed app where the AI pipeline runs end-to-end on real user input, from photo to ranked recipe. Going from "we don't really know how recipe-ranking works" to shipping a working embeddings-based ranking system felt like the biggest actual learning win of the whole project.

What we learned

We learned that in applied ML, the model is often the easy part — the ranking, filtering, and fallback logic around it is where a project becomes genuinely useful instead of just a cool demo. We also got real experience combining multiple AI techniques (generation, vision, embeddings) into a single coherent product rather than showcasing them in isolation, which pushed us to think about AI as a system, not a single API call.

What's next for Petugram: Smart Kitchen, Zero Waste Move from web prototype to a native mobile app Train or fine-tune our own recipe/vision models instead of relying fully on external inference APIs Expand the recipe and food database for broader cuisine coverage Add personalized recommendations based on a user's cook history and saved items Grow community adoption city by city to make food-sharing genuinely useful at scale Explore partnerships with restaurants, grocery stores, and sustainability organizations