Inspiration
Several of our friends and I had lab results come back with certain values flagged low or high, meaning our doctors wanted us to pay attention to what we eat as we felt dizzy and run down between studying. But the report was just a page of numbers where nothing connected "low iron" or "high sugar" to the food we actually had at home, and between classes and midterms, most of us weren't cooking much at all.
We also noticed how much food we bought with good intentions and then forgot about until it spoiled. Some of our friends skipped meals during exams, even though groceries were sitting in the fridge.
We want to solve the problem and build the direct bridge from the lab results to actual progress in restoring what was flagged so students like us won't feel overwhelmed when reading their lab results and motivated to take care of themselves.
What it does
PantryPulse turns a grocery receipt into a smart fridge which suggests recipes based on expiring foods and lab results, and turns a lab report into food focus areas, so flagged results lead to everyday meals.
- Scan a receipt. Photograph your groceries and the app lists the food items it reads. You review and fix the list before anything is saved.
- See your digital fridge. Every item gets an expiry date and a green, amber or red chip. The soonest-to-expire items come first, so you know what to use today.
- Upload a lab report (optional). The app reads your results, and you confirm each value against the original report. Results the lab flagged as low or high become plain-language focus areas to discuss with your doctor. For example, low hemoglobin and ferritin lead to "include iron-rich foods", where the system would suggest common foods that could restore those levels.
- Get recipes that fit. Recipes labeled with times and servings and are ranked by what expires first, then by your focus areas. A 15-minute scrambled egg and toast can use the milk with one day left.
What it doesn't do: it doesn't diagnose, treat, or estimate your lab values. Use the flags printed on your own report, and every suggestion is a general suggestion and should not be taken as medical advice. As an extra feature, we have also uploaded links to medical websites on each level for additional information. Expiry dates are estimates, throw out any food when in doubt. Specific foods without a common expiry date are left for manual input.
How we built it
PantryPulse has two parts: a design we prototyped first, and a Java engine that does the logic. We kept the AI to the jobs that need it, and used plain code for everything health-related.
Design
- Screens and flows of the expected finished project were designed in Figma, using one set of sample data (one receipt, one lab report) across every screen.
- A shared palette and type system, with expiry colors reserved for expiry chips and blue used for lab flags so the two never get confused.
- Later, our code is built with reference to what the Figma prototype looked like. ### Backend (Java)
- Receipt scan: the user uploads a photo to
/api/receipt. We check it is a JPEG, PNG or WebP, then send it to Google Gemini with a prompt that treats any text in the image as data, not instructions. Gemini returns a JSON list of food names. - Shelf life: a curated
shelf-life.jsontable maps each food to days remaining before it spoils. This is a lookup based on common known times for when food gets spoiled. Items with no match get a general 5-day estimate and a "needs review" flag. - Expiry and colors:: Red correlates to 0-2 days remaining (or expired), these are placed at the top to remind users to find a way to eat these before they spoil. Amber means 3-5 days remaining where it represents a warning with some wiggle room and green means any more than 5.
- Recipes:
/api/recipessends the pantry (each item with its days left) to Gemini and asks for a short recipe that prefers items expiring soonest. We validate the response before showing it and tell the model not to claim food is safe based on estimated dates. - Reliability: API keys are read from environment variables and never committed, responses are validated in code, and the demo can fall back to a sample pantry from
/api/pantry/sample. ### Not built yet A feature suggesting what foods to buy in your next grocery haul (apparent in Figma).
Challenges we ran into
- Getting reliable output from an AI model. During testing, "What should I cook?" returned a placeholder recipe called "Connection test" with water as the only ingredient. -Merge conflicts. Two of us edited the same controller with two different prompts, and the conflict markers broke the build. It taught us to split work by file, commit small, and pull often.
- Staying in scope. We started with prices, calories, caffeine, takeout receipts and daily meal logging. We cut all of them. Calories and intake can't be read from a receipt, and a blood test can't be estimated from groceries.
Accomplishments that we're proud of
-Learning how to implement Gemini and work into our working model with no prior experience in trying these types of things into our code. -Finishing an entire project within a time crunch of 24 hours with little breaks and paying our full attention on our first hackathon where we were used to spending weeks on a project. -As our first time using Figma, we're proud of our speed in learning how to work the program and creating a finished model that could be used to assist with building the platform.
What we learned
Use AI for reading and explaining, and plain code for anything that matters. Gemini is good at turning a messy receipt photo into a list of foods. Expiry dates and health focus areas need to be predictable and sourced, so those come from lookup tables and rules, not from a model's guess.
- Never trust AI output without checking it. The model returned fenced text, unfinished answers, and even a fake "Connection test" recipe. Validating every field before it reaches the screen turned a surprise on demo day into a visible error we can handle.
What's next for PantryPulse
- Add takeout and restaurant receipts. Many students eat out during exams, so we want the fridge to know about those meals too, and suggest what to cook next with that in mind. Menu items are harder to match to foods than groceries, so we'd start with simple cases and always ask the user to confirm.
- Compare reports over time. When a user uploads a new report, show the new and old values side by side, with the lab's own flags, to bring to their doctor.
- Shared fridges for roommates, so a household can see what's expiring and cook together.