SpoilSafe

From receipt to recipe, before it expires.

Comment

Inspiration

While brainstorming ideas for our project, our team noticed that we all faced a similar problem: food in our fridge would often spoil due to our forgetting expiration dates or never logging what we buy. So we created an app that saves your receipts, tracks expiration dates and notifies you when they're coming up, and gives you dishes to cook with the stuff you have.

What it does

SpoilSafe is a receipt-scanning application that uses OCR from Gemini to turn grocery receipts into organized, user-friendly, labeled data. We inform users of expiration dates and craft special meals out of the expiring groceries, which reduces users' food waste.

How we built it

We first designed the barebones UI of the app with Figma, eventually coding the frontend of the app with Next.js as we sought to have a working slate to work with. To port it over to iOS, we turned it into a Progressive Web Application (PWA) so users could access it like any app other on their home screen. With our team consisting of mostly beginners, we struggled to quickly learn how on being able to get Gemini being able to read the receipt, however with some debugging, we were able to solve it :)

Frontend:

  • Next.js adapted to iOS via Progressive Web Application (PWA)
  • Tailwind CSS for a clean, mobile-first design
  • Framer Motion for smooth animations that make the app feel polished
  • Figma to first plan our vision out

Backend:

  • Express.js backend handles AI image processing, recipe matching, and other API actions
  • PostgreSQL database to manage user data and authentication
  • Cloud Object Storage to store receipt images using an S3 compatible storage provider
  • Google Gemini analyzes receipt images and extracts structured data
  • Unified API management and context engineering for accurate extractions

Database & Security:

  • User authentication with middleware enforced, managed by Supabase (PostgreSQL)
  • Cloud storage for receipt images
  • Row-level security ensures users only see their own data

Mobile-First Design:

  • Framework adapted to iOS via Progressive Web Application (PWA)
  • SSR for faster first loads and up to date content delivery
  • Bottom navigation, responsive layout for any phone size, and fast loading times

Challenges we ran into

The biggest challenge was dealing with images, both on the API side (processing) and the storage side. We had to do a lot of debugging to get images sent properly to inference APIs, and even then, we had to experiment with multiple models in order to land on one that was reliable and cost effective (gemini-2.5-flash). For storage, we knew we wanted something like an S3 bucket, but since we were already using Supabase and Supabase Storage has an S3 compatible API (probably a wrapper of S3 itself), we went with that to have a more unified storage architecture.

Accomplishments that we're proud of

We're proud of how well the extraction works, which holds down the entire app and makes it really nice to use. We think this is something that can truly contribute to society in an impactful way, so we wanted to make it as easy to use as possible and I think we did that successfully.

What we learned

We learned how to:

  • Turn a Next.js web app into a PWA to the experience was optimized for mobile users
  • Interact with multiple APIs and host our own through a unified backend
  • Properly prompt for inference APIs in order to get consistent results
  • Create features from a user-first perspective (we tested this app a lot as we made it)

Built With

Share this project:

Updates