SmartCart AI: Qwen MemoryAgent
Track: MemoryAgent
Project history — what's new for this submission
This project began as SmartCart AI, originally built for the Google Cloud Rapid Agent Hackathon (submitted June 11, 2026), using Gemini 2.5 Flash via Vertex AI, Google ADK, and a Google Cloud Run backend.
For this submission, the project was substantially rebuilt for the Qwen Cloud Global AI Hackathon:
- AI backend migrated from Gemini/Vertex AI to Qwen Model Studio (
qwen-plusfor text,qwen-vl-plusfor vision) - Infrastructure migrated from Google Cloud Run to Alibaba Cloud Function Compute
- Vision-based receipt scanning — previously non-functional, built out into a working pipeline during this submission period
- User authorization — added authentication/authorization to the app for this submission, so pantry, recipes, and history are tied to a signed-in user
- Custom/saved recipes — new feature allowing users to save and customize their own recipes, persisted per user
- Recipe customization — new user-facing controls to adjust and personalize generated recipes
- Feedback form — new in-app feedback feature (delivered via Gmail SMTP), added to support live beta testing during this submission period
- Store-price and Firestore persistence layer — rebuilt from a non-functional stub into a working implementation
- Location detection rebuilt — fixed an IP-geolocation bug that defaulted users to the server's region (Singapore) instead of their own
- Extensive additional bug fixes found through live beta testing during the submission window
The React frontend and Firebase Hosting deployment were retained from the original project, but the core AI/agent logic, cloud backend, and several user-facing features (authorization, recipes, customization, feedback) are new for this track.
Inspiration
Grocery shopping is repetitive but personal — the same household preferences, dietary needs, budget constraints, and pantry state carry over week to week, yet most shopping tools treat every session as a blank slate. SmartCart AI is built to remember: pantry contents, meal history, preferences, and location persist across sessions so the agent gets more useful the more it's used.
What it does
SmartCart AI is an AI-powered grocery planning agent that:
- Generates weekly meal plans based on user preferences and current pantry contents
- Builds shopping lists automatically from meal plans, with a manual list mode
- Reads receipts via image upload (vision) to log purchases automatically
- Compares store prices and suggests the best-value stores nearby
- Optimizes shopping routes across multiple stores
- Breaks down nutrition per meal
- Suggests budget-friendly substitutions when a plan runs over
- Persists user memory — pantry state, preferences, meal history, customized/saved recipes, and store prices captured from scanned receipts — in Firestore, so returning users pick up where they left off
- Store prices are populated from receipts as users scan them; where real data isn't yet available, prices are mocked (see Known Limitations) to keep the demo functional end-to-end
How we built it
SmartCart AI runs on a cross-cloud architecture built specifically for this hackathon:
- Frontend: React, deployed on Firebase Hosting
- Backend: FastAPI, deployed on Alibaba Cloud Function Compute
- AI: Qwen Model Studio —
qwen-plusfor text generation (meal plans, shopping lists, substitutions),qwen-vl-plusfor vision (receipt reading) - Persistent memory: Google Firestore, storing pantry state, preferences, and meal history per user across sessions
- Location: Google Geocoding API with manual city/state fallback
The backend was migrated from an earlier Gemini/Vertex AI implementation to a fully Qwen-powered stack, with all LLM calls going through Alibaba's OpenAI-compatible Model Studio API.
Challenges we ran into
- Deployment tooling: Serverless Devs CLI failed due to an empty component registry, requiring manual deployment through the Alibaba Cloud Function Compute console instead.
- Silent bugs during live beta testing: several issues only surfaced under real usage — receipt images were being silently dropped in the vision pipeline, a store-prices repository was a non-functional stub never writing to Firestore, and shopping list mode crashed on edge cases with zero planned meals. All were caught and fixed during a live beta pass.
- Location defaulting to Singapore: IP-based geolocation was resolving to the Alibaba Cloud server's region (Singapore) rather than the user's actual location. Rebuilt location handling around manual city/state entry plus Google Geocoding.
- Recipe instructions truncating: Qwen responses were occasionally cut short due to too low a
max_tokensceiling for full JSON output — increased to accommodate complete responses.
Accomplishments that we're proud of
A fully functional cross-cloud deployment — AI inference and backend compute on Alibaba Cloud, frontend and data layer on Firebase/Google Cloud — working together as one coherent product, hardened through real live beta testing rather than a single demo run-through.
What we learned
Cross-cloud architectures surface failure modes that don't show up in single-cloud setups — regional defaults (like IP geolocation resolving to server location instead of user location) are easy to miss until real users hit them. Live beta testing before submission caught several bugs that a code review alone wouldn't have.
Known limitations
- Currency conversion is currently mocked. Prices shown in INR use a fixed approximate multiplier (USD baseline price × 80) rather than live India-region store pricing. This is a placeholder for demo purposes — real localized pricing data is a planned next step, not a bug.
- Store pricing and route data are demo-scale and not yet connected to live retailer APIs.
What's next
- Real localized pricing data per region/currency
- Expanded store and retailer integrations
- Deeper personalization from accumulated memory (long-term preference learning, not just session state)
Log in or sign up for Devpost to join the conversation.