SmartBasket

SmartBasket fights grocery inflation. Upload a receipt, and our AI tracks cross-store prices to build an optimized 'Split Trip' itinerary that maximizes your household savings automatically.

Comment

Inspiration

Grocery prices are simply too expensive and I hate trying to find where is the cheapest to buy grapes or apples

What it does

SmartBasket turns ordinary supermarket receipts into dynamic, money-saving itineraries. Users simply upload a picture of a receipt. The application automatically extracts the items, tracks regional prices over time, and builds an optimized shopping plan. With a clean toggle interface, users can choose between visiting a single cheapest store or a "Smart Multi-Store Split Trip"—which shows exactly how much cash they save by dividing their list between competing grocery locations.

How we built it

I engineered a decoupled, production-ready stack using a combination of edge delivery and persistent backend queues:

  • Presentation Layer: A responsive dashboard built with Next.js, scaffolded via Vercel v0, and deployed on the Vercel Edge Network for instant page loads.
  • Persistent Backend Hub: Hosted entirely on an AWS EC2 instance. Both our FastAPI server and background worker process run 24/7 as concurrent systemd daemons.
  • Asynchronous Buffer: When a receipt is uploaded, the API checks a unique SHA-256 image hash to prevent duplicate parsing. It stores the file bytes in Amazon S3 and pushes a lightweight task payload into an Amazon SQS Queue to unblock the frontend immediately.
  • AI Computer Vision Pipeline: A background worker daemon continuously polls SQS, retrieves the image from S3, runs text extraction using OpenCV and EasyOCR, and leverages Groq (Llama 3.3-70B) to automatically organize chaotic text lines into a structured JSON schema.
  • Relational Data Foundation: Data is securely committed using server-pooled connections to an Amazon Aurora PostgreSQL cluster enforcing strict production SSL encryption.
  • Fuzzy Catalog Matching: To reconcile sloppy receipt abbreviations (like matching "RDSD GRP" to "Red Seedless Grapes") in milliseconds, we utilized Aurora's native Trigram Similarity Extension (pg_trgm) to evaluate mathematical similarity() index scores directly within our database queries.
  • Live Streaming Alerts: We built persistent Server-Sent Events (SSE) channels mapped to an in-memory queue to push live alerts straight to a user's dashboard the exact second our background ingestion system flags a cheaper alternative for an item in their basket

Challenges we ran into

  • Avoiding Web Timeouts: Executing heavy computer vision and large language model workflows directly inside a standard web request easily crashes due to gateway timeouts. We solved this by splitting our infrastructure completely: saving file payloads to S3 and offloading task queues to SQS lets the frontend stay hyper-responsive while the EC2 worker processes heavy computer vision tasks safely in the background.
  • The Supermarket Shorthand Trap: Supermarkets print the exact same product with entirely different abbreviations and typos. Introducing slow, heavy external search indexes wasn't an option. We solved this cleanly at the data layer by performing native trigram similarity() query scans directly within our Aurora PostgreSQL instances to map messy labels instantly.
  • Taming Real-Time UI Synchronization: On a fresh page reload, our real-time SSE stream would eager-pull unread notification logs from the database before the front-end DOM finished its mounting cycle, causing notifications to disappear or double-fire. We separated concerns: live alerts stream over dedicated SSE sockets, while page initializations use a safe REST query backed by an explicit user-acknowledged /notifications/read status endpoint to sync states securely.

Accomplishments that we're proud of

  • Building a fully functioning, highly responsive asynchronous image processing engine and AI data pipeline over a single weekend.
  • Engineering a production-grade, decoupled stack where computationally intense tasks are safely isolated from the client thread to protect the user experience.
  • Cracking the product mismatch problem natively at the database layer using high-speed trigram lookups rather than relying on bulky application-layer code loops.

What we learned

  • We discovered that you don't have to choose between development speed and stability—combining Vercel v0 for rapid front-end iteration with the enterprise-grade relational power of Amazon Aurora provides a production-grade foundation on day one.
  • We learned that relational databases are incredibly versatile, and that utilizing native extensions like pg_trgm can clear data-matching bottlenecks faster and more elegantly directly inside the database tier.

What's next for SmartBasket

  • Crowdsourced Price Maps: Allowing users to securely opt-in to share anonymous pricing updates from their receipt scans, transforming every individual upload into a shared community savings map.
  • Geofenced Travel Optimization: Integrating mapping APIs to calculate actual driving distances between supermarkets, automatically factoring fuel costs into the optimization logic to ensure a multi-store split trip is truly worth the drive.
  • Direct Checkout Integration: Building cart integrations with major online grocery delivery APIs, allowing users to export their optimized baskets straight to their favorite supermarket checkouts with a single click.
Share this project:

Updates