Inspiration The idea was born from a common frustration: grocery receipts are often cryptic, printed on thermal paper that fades, and contain data that is difficult to track manually. I wanted to build a bridge between traditional physical shopping and modern data analytics, allowing users to understand their spending habits without the friction of manual data entry.
What it does SmartCart AI is a web application that transforms a simple photo of a supermarket receipt into an interactive dashboard. Using Generative AI, the app extracts specific details—product names, quantities, unit prices, and categories—even from wrinkled or poorly lit receipts. It then processes this data to provide users with meaningful metrics about their consumption patterns and price fluctuations.
How we built it The application is built on a modern, scalable stack:
Frontend: Developed with Angular, providing a responsive and fluid user interface for uploading images and viewing charts.
Intelligence: I integrated the Google Gemini API (via AI Studio) to handle the OCR and data structuring. Gemini’s multimodal capabilities allowed me to skip traditional, rigid OCR engines in favor of a more flexible "natural language" extraction.
Backend & Logic: Using Python, the application parses the AI’s JSON output, validates the totals, and prepares the data for visualization.
Challenges we ran into One of the biggest hurdles was dealing with the inconsistency of physical receipts. Variations in font, ink quality, and "noise" (like logos or creases) often lead to errors in traditional extraction. Fine-tuning the system prompt for Gemini was crucial to ensure that even "messy" data was returned in a consistent, structured format that the frontend could reliably display.
Accomplishments that we're proud of I am particularly proud of the accuracy achieved. By leveraging LLMs, the app can intelligently "guess" the category of a product (e.g., recognizing that "Whole Milk 1L" belongs to "Dairy") without a pre-defined database of millions of products. Successfully deploying the system and seeing it turn a blurry photo into a clean data set was a major milestone.
What we learned This project reinforced the power of multimodal AI. I learned that the future of data entry isn't just about reading text, but about understanding context. I also deepened my experience in managing asynchronous operations between an Angular frontend and high-latency AI API calls to ensure a smooth user experience.
What's next for SmartCart AI: From Your Receipts to Smart Decisions The roadmap for SmartCart AI involves:
Price Comparison: Tracking the price of the same item over time to alert users of inflation or deals.
Personalized Budgeting: Predictive analytics to tell users how much they are likely to spend next month based on history.
Inventory Management: Automatically adding scanned items to a "digital pantry" that suggests recipes based on what you just bought.
Log in or sign up for Devpost to join the conversation.