PantryAgent

Home
Scan
Shopping
Meal Plan(Diet Log)
Pantry
Recipe
link to our website

Inspiration

Food waste is a surprisingly common problem in everyday life. The USDA estimates that 30-40% of the U.S. food supply is wasted. Yet much of this waste does not happen because families do not care. It happens because everyday food decisions are disconnected: people forget what they bought, lose track of what expires next, struggle to turn available ingredients into meals, and buy items they already have.

Confusion around food date labels makes the problem worse. A 2025 national survey from the Harvard Law School Food Law and Policy Clinic, ReFED, and the Johns Hopkins Bloomberg School of Public Health found that 43% of U.S. consumers always or usually discard food near or past its label date, while 88% do so at least occasionally.

Our team saw that recipe apps, grocery lists, pantry trackers, and meal planners usually operate as separate tools. We wanted to connect those decisions into one continuous system that helps households use food before buying more.

That idea became PantryAgent: an AI-assisted food management system that turns pantry inventory into actionable, low-waste meal decisions.

Sources: USDA Food Loss and Waste and ReFED's 2025 national survey.

What it does

Receipt to pantry workflow

PantryAgent helps users manage food from purchase to consumption.

Users can photograph a grocery receipt, and Claude Vision extracts edible items, quantities, categories, and estimated shelf lives. Before anything is saved, users can review, edit, or remove the extracted items. The pantry then sorts food by expiration urgency instead of alphabetically.

Users can also import public recipe-blog URLs. Browserbase converts the original page into structured ingredients and instructions, allowing trusted recipes from the web to participate in the same planning workflow as saved recipes.

PantryAgent recommends meals that use soon-to-expire ingredients first. Its weekly planner considers pantry quantities, recipe requirements, imported recipes, and meals the user selected manually. It flags likely shortages and adds only missing ingredients to the shopping list.

The loop continues after planning: marking a shopping item as bought adds it to inventory, while marking a meal as eaten records the meal and deducts matching ingredients from the pantry. Nutrition estimates support the workflow, but waste reduction remains the primary goal.

How we built it

We built PantryAgent using modern web technologies, a structured database, and AI-powered automation.

The application uses Next.js, React, TypeScript, and Tailwind CSS to deliver a clean, mobile-first experience. Supabase and PostgreSQL store pantry inventory, recipes, meal plans, shopping lists, dietary logs, and web-extraction records.

Claude powers grocery receipt understanding, meal-plan generation, and nutrition reconciliation. Browserbase serves as our web-action layer: it retrieves recipe content through Fetch and escalates to a real browser with Stagehand when pages require JavaScript, scrolling, popup handling, or interaction.

When a user submits a recipe URL, Browserbase extracts the original page into a consistent JSON structure. The system validates that essential information, such as ingredients and instructions, is present before saving it. For missing nutrition data, Browserbase searches for supporting sources, while Claude reconciles the evidence with ingredient quantities and serving sizes. Confidence labels, source links, and action logs make the process more transparent and reduce unsupported AI assumptions.

Our meal-planning engine combines pantry quantities, expiration dates, recipe requirements, and user-selected meals to prioritize food that should be consumed first. It identifies shortages, adds only missing ingredients to the shopping list, returns purchased items to inventory, and deducts ingredients after a meal is completed.

We also used Pika as our creative production layer. Pika supported the visual development of our website, team logo, and UI assets. Our final demo video, including AI-generated team presenters, voice and facial synchronization, supporting visuals, transitions, captions, and editing, was also generated and assembled using Pika.

Challenges we ran into

One major challenge was extracting reliable structure from inconsistent recipe websites. Ingredients can omit quantities, instructions may be buried inside long articles, and important content may sit behind popups, collapsed recipe cards, or lazy-loaded sections. A single extraction method was not reliable enough, which led us to build the Fetch-first validation and Stagehand fallback pipeline.

Another challenge was deciding how much AI should infer. Receipt shelf life, recipe quantities, and nutrition values can all contain uncertainty. We added editable receipt results, schema validation, source-backed nutrition evidence, confidence labels, and deterministic calculations instead of treating the first model response as unquestionable truth.

The hardest product-design challenge was not any individual AI call. It was making inventory, recipes, planning, shopping, and meal completion behave as one stateful loop. We repeatedly refined the data contracts and transitions so that each action changes what the next recommendation sees.

Accomplishments that we're proud of

We are proud that PantryAgent is an end-to-end product rather than a collection of disconnected AI features.

We built a receipt-to-pantry workflow with human review, an expiry-aware inventory, a Browserbase recipe importer with a real browser fallback, an AI-assisted weekly planner, evidence-aware nutrition estimates, and a shopping loop that returns purchases to inventory and deducts ingredients after meals.

Most importantly, the product connects those capabilities around a clear social-impact goal: helping households waste less food and avoid unnecessary repeat purchases.

Ethical considerations

We treat AI as an assistant, not an authority. Users review receipt results before saving them. Shelf-life and nutrition outputs are explicitly labeled as estimates and should not be treated as food-safety or medical advice. Nutrition results expose evidence, source links, and confidence instead of hiding uncertainty.

PantryAgent can prepare shopping lists but does not automatically purchase or check out. The current hackathon build uses a single demo user; a production release would require account isolation, least-privilege database access, explicit image-retention controls, and clear consent for any personal data processing.

What we learned

We learned that useful AI products depend less on one impressive prompt than on reliable data pipelines, validation, human checkpoints, and state transitions.

We also learned to match the tool to the task. Browserbase Fetch is the efficient path for simple pages, while Stagehand is valuable when the web becomes interactive. Claude is strongest when reasoning over structured context, and deterministic application code should perform calculations and state updates whenever possible. Pika is a flexible creative toolkit for generating images and videos, editing existing footage, and turning raw assets into polished visual stories.

Most of all, users care about whether the entire workflow reduces effort and supports better decisions, not how many separate AI features appear in the interface.

What's next for PantryAgent

We see PantryAgent as the foundation for a larger food management ecosystem.

Future improvements include direct integration with grocery delivery services such as Amazon Fresh and DoorDash, automatic pantry synchronization after grocery purchases, computer vision-based ingredient recognition from fridge photos, and personalized nutrition recommendations for goals such as weight loss, muscle gain, or dietary restrictions.

We also want to improve expiration prediction using food-specific shelf-life models and build AI agents that automatically update meal plans as pantry inventory changes.

Our long-term vision is to create a fully automated kitchen assistant that helps users reduce food waste, eat healthier, and spend less time managing groceries and meal planning.

By connecting inventory, recipes, shopping, and nutrition into one intelligent system, PantryAgent makes sustainable eating easier, smarter, and more effortless.