Welcome Page upon app launch
Sign In Page
Phone Number Verification
My Pantry Page
Integrated GoogleMaps for local donation places
Text received when sharing food with contacts
Web Application Page (Community Food Search)

PantryPal

Inspiration

According to the Department of Agriculture, 30-40 percent of the United State's food supply is wasted. A large contributor to this statistic is food spoilage. We have all experienced forgetting about a gallon of milk only to be taken back by the stench when going to pour a glass. While food spoilage can occur at all stages of production, many Americans see first hand the problem with spoilage in their own pantry. With busy lives, many people forget to check expiration dates, are unsure of what to make with leftover ingredients, or simply cannot eat all of the food in time before it expires. We sought to solve this exact problem with PantryPal.

What it does

PantryPal's main focus is on keeping track of expiration dates and suggesting ways to use or donate unused food to reduce food waste. Our app has four main functionalities: pantry tracking, recipe suggestion, local donation, and global food posts.

Pantry

The "pantry" stores all of the user's food. To get started, users can scan their grocery receipts and automatically add all of their food to their virtual pantry. Compared to similar food tracking apps on the market that require scanning barcodes, PantryPal requires scanning only once to add all foods. This eliminates the tedious and impractical aspects of current apps. PantryPal also stores and displays the estimated expiration date, and soon to expire foods are color coded for the user. When a food in the user's pantry becomes high-priority (within 3 days of expiration), the user receives a notification on their phone that the food is soon to expire. Users can also remove food items from the pantry and post an offer for the food item to either their contacts or public.

Recipes

If a user has soon to expire food and is unsure of what to make with it, PantryPal will give recipe suggestions using food already in your pantry. PantryPal generates many recipes and provides both the ingredients and directions for preparation and cooking.

Donation Map

If a user does not have time to cook or simply cannot eat their food before it expires, PantryPal displays an interactive map to the user with pins on all local donation centers. Here, users can easily find and navigate to food donation centers near them to donate any extra or soon to expire food they have.

Community Food

Another alternative to sharing your food includes sending an offer to those in your community or phone contacts. With PantryPal, users can notify friends or family from their phone contacts to let them know about the available food items for sharing. Users can also choose to post their food offer publicly. Users can both post public offers and search through public offers of food near them available for pickup.

PantryPal aims to limit food waste by tracking expiration dates for users, providing recipe suggestions, and connecting users to their community to share food.

How we built it

Frontend:

React Native
Figma was used to design each frame and the flow between pages as well as to make graphics

Backend:

Nest.js for MVC structure, HTTP server
Custom framework built on top of Nest.js for admin dashboard, routing layer, and error handling
MySQL for database to store user data, Prisma as ORM to manage interface
C++ integration with Tesseract OCR to recognize text from receipts
NotificationAPI for sending SMS notifications
JSON object pilfered from FDA website for food expiration times, supplemented by additional manual research
Spoonacular API for recipe/ingredient information
GeoAPIfy for reverse geocoding lookup to localize results for users

Challenges we ran into

When we first started, we ran into lots of issues when scanning a receipt. We realized the array was too small because of the discrepancy between the number of confidences and words that are available since there were more words than confidence values. To fix this issue, we had to collapse the spaces between words.
When trying to set up SMS text verification and notifications, we were struggling to find a free service that did not require multi-day verification to use. We eventually found a service that we could use immediately after research.
It was tough to find data on shelf life information for different types of foods. We tried a lot of different APIs but all of them had no info about shelf life information. After extensive research, we discovered a JSON dataset on the USFDA website.
We wanted to bring our Figma assets to our app as SVGs but React Native has limited support for SVGs, so we had to manually animate one of our animations even after creating it as an SVG using SVGator. This costed us lots of time during the hackathon
It was hard at first to think of every edge case for the Figma design. For example, what should we display to users when someone has no food in their pantry? This added the number of graphics needed which increased the time spent designing. We were able to craft cool designs and graphics for each edge case and identify more edge cases when testing the mobile app.

Accomplishments that we're proud of

We are proud of getting the receipt scanner to work and scan accurately most of the time. We are proud of our OCR results and how we removed noise. We were able to use text editing boxes after the scan to allow users to fix small changes in the scan.
We are proud of using SMS notifications to alert users of soon-to-expire food and using a six digit code verification for users upon sign in.
Creating both a mobile and web app during the weekend.
Animating the welcome page's three icons for an impactful first impression.

What we learned

We learned how to use easing on SVGs to change the rate of acceleration in our asset animation.
We learned how to implement SMS notifications and verification and how to use Tesseract to scan text from an image for the receipts.
We learned how to integrate Google Maps into a mobile app.
We learned how to sync contacts from user's phone to the app.

What's next for PantryPal

We envision additional features such as not allowing the same food to be posted globally twice, recipe filters, including dietary restrictions, only food in the pantry, or soon-to-expire, and a secure in-app message and rating system for safer public food transactions. We also hope to make PantryPal more small business friendly by storing quantities of food items and adding the ability to give out dishes to the community rather than just ingredients.