PantryPilot

Don't let groceries rot in the back of your fridge ever again.

Comment 2

Inspiration

Food insecurity remains a big challenge to solve in many parts of the world. Despite that, there is so much food wasted across households, grocery stores and restaurants. The United States itself wastes around 29% of its food (ReFED, 2026), at a time where 13.5% of its households are food insecure (U.S. Department of Agriculture, Economic Research Service, 2026).

Households are responsible for 33.5% of all surplus food waste in the U.S.A, at a cost of USD 728 per person, per year (ReFED, 2026). Most reasons are due to ingredients being spoiled after forgetting about them, over purchasing (Blondin et al., 2025), and inability to meal-plan in an effective manner. Furthermore, buying ingredients not commonly used in their regular diets can let them sit unused till they eventually have to be thrown out (Gunders & Bloom, 2017).

We made PantryPilot to take a small step towards reducing household food wastage. Let PantryPilot fly your forgotten-about groceries away from Destination: Garbage, straight to your plate.

What it does

Pick a cuisine (or let it surprise you!), select any dietary restrictions that apply. Then when you're ready, just take a short video of your fridge/pantry, or up to 4 photographs. The app creates an inventory based on what it detected (manually addable, updateable and removable), and then provides a recipe based on your preferences, with a generated, representative image to whet your appetite. You can save these recipes locally, and add any notes to them. It automatically tags items that might require to be used up soon, and lets you know what percentage of such items were used up in the recipe!

Don't have a well stocked fridge? Trying a new cuisine but missing 1-2 items? Fret not, the app will let you know what is missing, along with the price and location (closest stores first) of the item.

This approach is grounded in behavioral science. Reducing friction and information-search costs can make intended behaviors easier to initiate (NASEM, 2023), while narrowing a complicated choice into a small number of relevant recommendations can reduce the conditions associated with choice overload (Chernev et al., 2015). Flexible "use-up" recipe interventions have also produced substantial reductions in self-reported household food waste, partly by increasing participants' perceived ability to turn unused ingredients into meals (Cooper et al., 2023). In a study involving 239 users, Starke et al. (2021) found that the visual attractiveness of recipe images affected recipe selection. Participants were more likely to select healthier recipes when those recipes were accompanied by visually attractive images and ranked prominently in the results.

DOWNLOAD: https://github.com/codexblack/PantryPilot/releases/tag/android-apk-latest (ANDROID ONLY)

How we built it

We thought of the basic building blocks, organization of projects, along with an initial UI. This was provided to GPT-5.6 to create a visualized and structured plan.

Downloading ChatGPT Windows Desktop app, we used the Codex agent to read this plan, and create the full application end-to-end, one step at a time. It helped put together the Expo screens, FastAPI endpoints, inventory scan flow and recipe generation flow, instead of us having to build all the pieces separately first. As we tested the one-shot generated app locally, we kept adding features to it one by one, and suggesting UI changes. Codex was then used to look through the relevant files, make focused changes, and run the relevant type checks, linting and tests before moving to the next feature.

For most features, we used GPT-5.6 Terra Extra High. For a full refactor and thinking about complex architectural decisions to optimize processing time, we used GPT-5.6 Sol Ultra. This was especially useful for tracing a user action from the React Native app through the FastAPI backend, and for keeping the mobile app and API contracts in sync as features were added.

React Native with Expo was chosen as a cross-platform and simple method to build a mobile app. We created a backend using FastAPI, responsible for passing filtered video frames or images to OpenAI's models, while structuring the prompt. The image for the dish is generated by gpt-image-2. Recipe details are returned first, while the dish image is requested separately, so the user can start looking at the recipe while the image is still being generated.

Using Codex to set up GitHub Actions for CI/CD, we deploy the backend in a docker container on Google Cloud Run. Firebase App Check is used to make sure the API is only reachable by our registered client app. We don't store any images or videos on our side, the API remains stateless, and uploaded media is kept only temporarily while it is being processed.

Challenges we ran into

Image generation takes a long time, and it was difficult to think up how to dismiss the recipe before the image finishes loading, while keeping it loading in the background, to be visible when ready. GPT-5.6 Sol helped make this asynchronous.

Connecting the built app to the backend securely was a source of headache, that Codex with MCP enabled helped take care of after some prompting. It helped us work through the Firebase App Check setup, while making sure the backend checked the client token before allowing the protected API requests. I also do not have an Apple machine, so the app actually could not be compiled for both platforms, only Android for the time being.

Accomplishments that we're proud of

The app fully works, is deployable, and gives useful results. Not bad for only a day to get the first version out, followed by a few days of adding extra bells and whistles, considering we had time and credits left.

What we learned

It was my first time using Codex, and by the end I was quite comfortable with it. I always made sure I explicitly approved every time it wanted to do something. Also learnt getting set up with React Native and Expo, with a full CI/CD pipeline in GitHub Actions. I also learnt that being specific about the feature, the files to inspect, and how to test it made the back-and-forth with Codex much more useful.

Alongside this it was also interesting and demoralizing to read about the economics of food wastage, which was during the initial research phase.

What's next for PantryPilot

We are planning for a limited release after the judging concludes, to ensure consistent, long term correctness of the ingredient extractions in different kitchens with different purchasing habits. Since the infrastructure costs money, a full-scale release is not advisable without doing a full cost-to-utility ratio.

References

Built With

Share this project:

Updates