• 

  Dashboard to show live action of the agent

• 

  Every kitchen food consumables and produce available in the kitchen

• 

  Possible meal to be made with available kitchen inventory, also option for delivery to considerd in meal planning.

• 

  Food allergens and item disliked to tailor meals suggestion

• 

  Automated orders placed and confirmation to be done in order to update pantry.

• 

  Control panel of the agent, how decisions of the agent are configured

• 

• 

  Agents history

PantryPilot — A Kitchen Agent That Thinks Ahead

Inspiration

PantryPilot was inspired by a problem I kept running into in my everyday life: deciding what to eat, especially when I was sick and needed healthier meals.

I often found myself relying on takeout because figuring out what to cook felt like another task to manage. I had to check what was available in the fridge, remember what was about to expire, figure out which ingredients could work together, and consider my food preferences, allergies, and dislikes.

Despite having food at home, ingredients would sometimes go stale or expire simply because I forgot about them or didn't know what to prepare.

I wanted to build something that could take that mental load away — not just another recipe app, but an intelligent kitchen assistant that actively helps manage food, meals, and replenishment.

What We Built

PantryPilot is an autonomous kitchen agent that helps users manage their kitchen proactively.

Rather than waiting for users to decide what they need, PantryPilot monitors available stock, tracks expiry and freshness, suggests meals based on available ingredients, and identifies when essential items need replenishing.

Its key capabilities include:

• Smart Inventory Management: Keeps track of kitchen supplies and identifies items running low or approaching expiry.
• Personalised Meal Planning: Recommends meals based on available ingredients, dietary preferences, allergies, dislikes, and food preferences.
• Intelligent Substitutions: Suggests alternative ingredients when something is unavailable.
• Bounded Autonomous Shopping: Can initiate replenishment through existing stores, but only within spending limits and purchasing rules explicitly approved by the user.
• Purchase Verification: Tracks orders and confirms what was purchased, delivered, or remains outstanding.
• Transparency and Control: Shows users what the agent is doing, why it is doing it, and how their money is being spent.

How We Built It

We approached PantryPilot as an agent-driven system rather than a conventional meal-planning application.

The idea was to give the agent a continuous understanding of the kitchen's inventory and enough context to make useful decisions.

The workflow follows a simple cycle:

1. Observe: Gather information about available ingredients, quantities, and expiry dates.
2. Understand: Consider user preferences, dietary restrictions, allergies, and current needs.
3. Decide: Recommend suitable meals, identify missing ingredients, and determine what needs replenishing.
4. Act: Initiate approved purchases through connected retailers, respecting predefined budgets and purchasing constraints.
5. Verify: Check purchase and delivery outcomes and update the inventory accordingly.

A key design principle was keeping autonomy within boundaries. PantryPilot should be proactive, but it should never have unrestricted permission to spend a user's money.

Challenges We Faced

One of the biggest challenges was balancing automation with user control. An agent that can make purchases needs clear spending limits, permission boundaries, and a reliable way to communicate its actions.

Another challenge was making meal recommendations genuinely useful. Having ingredients available does not automatically mean they make a suitable meal. Recommendations need to account for allergies, dislikes, dietary requirements, ingredient combinations, and substitutions.

Inventory accuracy also presents a challenge. Items can be consumed, forgotten, spoiled, or replaced without the system knowing immediately. Keeping the inventory synchronised with actual kitchen stock is essential for reliable decisions.

Finally, integrating external retailers introduces additional complexity, particularly around product availability, order status, substitutions, and delivery verification.

What We Learned

Building PantryPilot reinforced an important lesson about autonomous agents: intelligence is not just about making decisions; it is about making decisions responsibly and verifying the outcomes.

We also learned that useful automation starts with understanding people's everyday frustrations. A good kitchen agent should not simply tell someone what to eat. It should reduce the effort involved in planning, help prevent food waste, and make routine household decisions easier.

Our vision for PantryPilot is simple: less time managing the kitchen, less food wasted, healthier everyday choices, and more confidence in letting an AI agent handle routine tasks within boundaries set by the user.

Built With

• next.js
• paypal
• postgresql
• prisma
• typescript