Inspiration
Food is often wasted not because people buy too little, but because they forget what they already have and which products should be used first.
Intelligent Pantry was already a working Android application available on Google Play before OpenAI Build Week. During the event, we extended the existing product with a new safety-first feature: the Pantry Rescue Agent.
The new agent was designed, implemented, tested, and prepared for demonstration in approximately two days with Codex and GPT-5.6 Terra.
In that same two-day sprint, Codex helped us audit and prepare the development environment, build the backend and mobile experience, create automated tests, diagnose emulator problems, and prepare the final competition materials.
Google Play:
https://play.google.com/store/apps/details?id=pl.inteligentnaspizarnia.smartpantry
The version currently available on Google Play confirms that Intelligent Pantry is a real working product. The Pantry Rescue Agent created during Build Week has not yet been deployed to the store version.
What it does
The Pantry Rescue Agent analyzes products stored in the pantry together with their expiry dates.
It:
- excludes products that are already expired,
- prioritizes food that should be used first,
- creates a practical plan containing up to two meals,
- identifies missing ingredients,
- allows the user to select suggested shopping items,
- updates the shopping list only after explicit confirmation.
The central safety principle is:
AI proposes. The user decides. The app safely performs only approved actions.
The model never modifies pantry data or the shopping list directly. It returns a structured proposal, while a deterministic application layer performs only the actions explicitly approved by the user.
How we built it
The mobile application is built with Flutter and Dart.
The agent backend uses a Firebase Callable Function running locally through the Firebase Functions Emulator. The function calls the OpenAI Responses API with GPT-5.6 Terra and requests a strict structured output named Rescue Plan 1.0.
The response is checked using:
- JSON Schema validation,
- semantic validation,
- product eligibility checks,
- expiry-date rules,
- quantity rules,
- request-size and response-size limits.
Expired products and items with a quantity of zero are excluded before data is sent to the model.
The backend uses store: false, does not log prompts, responses, product names, or secrets, and keeps the OpenAI API key outside the Flutter application.
The agent can propose shopping-list changes, but it cannot execute them itself. Selected items are written by deterministic application code only after the user confirms the action.
How Codex helped
Before Build Week, I had not used Codex.
During the event, Codex became the engineering partner for the complete feature. It:
- audited the development computer,
- installed and configured Flutter, Android SDK, ADB, Node.js, Firebase CLI, FlutterFire, and FFmpeg,
- analyzed the existing application architecture,
- created an immutable pantry model and repository layer,
- implemented the Firebase Functions backend,
- integrated GPT-5.6 Terra,
- added structured-output and semantic validation,
- built the complete Flutter user interface,
- implemented safe shopping-list updates,
- created automated Flutter, Functions, widget, and integration tests,
- diagnosed Android Emulator rendering problems,
- prepared demonstration data, screenshots, recordings, subtitles, and the final competition video.
Codex did not merely generate isolated snippets of code. It helped build, test, debug, secure, document, and present a complete extension to an existing product.
This compressed work that could normally take many days or weeks into approximately two days of focused development.
A surprising Codex moment
One of the most surprising moments happened during visual testing.
We took a screenshot of the Android Emulator because something looked wrong on the screen. Before we had written a detailed bug report, the screenshot became available in the active Codex session and Codex immediately analyzed the visible artifacts.
It correctly distinguished the issue from a Flutter UI overflow, identified it as an emulator GPU and Impeller rendering problem, and switched the demonstration runner to software rendering without changing the application logic.
This changed our workflow. Instead of describing every visual problem manually, we could provide direct visual evidence and let Codex determine whether the likely cause was the interface, the emulator, or the data.
It showed us that Codex could act not only as a coding assistant, but also as a visual tester and debugging partner working with the broader development environment.
Testing
The final project passed:
flutter analyze,- 40 Flutter tests,
- 38 Firebase Functions tests,
- a complete Android end-to-end flow using the Functions Emulator.
A real GPT-5.6 Terra request was also executed successfully.
The live response:
- passed JSON Schema validation,
- passed semantic validation,
- excluded expired food,
- did not treat shopping-list items as owned products,
- left the pantry unchanged,
- returned a valid two-meal rescue plan.
The production-oriented safeguards include request limits, controlled timeouts, zero automatic retries, response-size limits, limited function concurrency, a feature kill switch, and secret isolation.
Demonstration transparency
The complete shopping-list approval flow shown in the video uses a deterministic mock so that the entire safety process can be demonstrated reliably and consistently.
A separately labelled section of the video shows a real GPT-5.6 Terra response.
The real model response did not require additional shopping items. For that reason, the video does not pretend that the live response produced actions that were not actually returned.
The competition backend runs locally through the Firebase Functions Emulator. The Pantry Rescue Agent has not been deployed to production Firebase and is not yet included in the Google Play release.
Challenges
The main challenges were:
- preserving compatibility with the existing pantry storage format,
- ensuring that owned products could never be accidentally converted into shopping-list items,
- preventing the model from directly mutating application data,
- validating every structured response,
- safely excluding expired products,
- configuring Firebase App Check and local emulators,
- separating secrets from the mobile application,
- removing GPU rendering artifacts from Android Emulator recordings,
- clearly separating the deterministic safety demonstration from the live API result.
What we learned
The most important lesson was that an AI agent should not only be useful, but also predictable, transparent, and controllable.
Structured output alone is not enough. The application must still validate the response, enforce business rules, reject unsafe data, and require user confirmation before performing consequential actions.
We also learned how much Codex can accelerate work on an existing real-world product. It helped with the entire engineering process: environment preparation, architecture, implementation, testing, debugging, security, documentation, and presentation.
The visual debugging moment was especially important. It showed that the most effective workflow was not always to explain a problem in technical language first. Sometimes direct visual evidence allowed Codex to recognize the issue and select the correct layer to fix without unnecessary changes to the application.
Potential impact
Intelligent Pantry can help households:
- reduce food waste,
- save money,
- make better use of products they already own,
- plan meals around real expiry dates,
- avoid unsafe recommendations involving expired food,
- keep the final decision in the hands of the user.
The same safety-first approach could later support:
- family meal planning,
- dietary preferences,
- household budgets,
- personalized shopping recommendations,
- local food-sharing systems,
- cooperation with food banks or community organizations.
What is next
The next steps are:
- production deployment of the Firebase Function,
- production App Check and Play Integrity configuration,
- controlled testing with real users,
- adding optional dietary preferences,
- improving explanations of why individual products were prioritized,
- adding more personalized rescue-plan settings,
- releasing the Pantry Rescue Agent in a future Google Play update.
Log in or sign up for Devpost to join the conversation.