BuyBetter

Front page of the website
List of all receipts along with option to add a new one
Loading screen for receipt
Receipt details screen showcasing possible alternatives along with links and maps to assist

Inspiration

The financial focus of many of the challenges got us thinking about potential applications for college students, and one big issue that I find me and many of my friends making is impulsive buying. Spending an unnecessary amount on many purchases rather than finding the cheaper alternative. This project is directly geared towards that, and we hope that it can make a decent impact.

What it does

BuyBetter takes your receipts and finds online stores, physical locations near you, and online resources that can assist you in making cheaper and more financially responsible alternatives to your purchases.

How we built it

With a Node.JS and Express backend along with a vanilla JS frontend, we sticked to the less complex development route that we were more familiar with.

Backend

Using Google Cloud's Vision model for OCR, we were able to read the text on the receipt and throw it into OpenAI's gpt-4o model to output alternative purchases or activities that would result in a lesser financial cost. The LLM message chain first finds the purpose behind the purchase, along with several alternatives, and creates a Google/Google Maps/Amazon search query that would help the client find places to purchase their receipt alternatives. By utilizing the Google Custom Search Engine API along with Google Maps Embed API, we were able to display the information in a simple user-friendly manner.

Frontend

Although the frontend was built using purely HTML, CSS, and vanilla JS, we were able to create a sleek UI design and a nice looking webpage. Part of the frontend functionality is that we store the receipts in order to be able to come back to. This was built using the window.localStorage object, storing a list of all receipt IDs, along with all of the receipts. This localStorage came in handy when trying to coordinate processes and data across different pages on the website. Another frontend capability that we incorporated was response pinging. Once a receipt file was uploaded, the backend would respond with the given ID of that receipt. The frontend would then ping the server every 500ms to watch the progress and status of the receipt's path through the backend. This was very useful, allowing us to send progress updates to the user, along with notifying the frontend in case of an error or issue.

Challenges we ran into

A few of the more prominent challenges came with connecting to the several APIs that we used. The Google Cloud Vision API in particular came with slightly confusing API authentication procedures, and it took about an hour to understand where my issues were spawning from. Another issue that we came across is that Amazon search API was difficult to set up and obtaining an API key to test with and work with needed approval. The workaround for this issue was using the Google Search API with the query "amazon " + query to take the results that come from amazon.com.

Accomplishments that we're proud of

Something that we are proud of is the short timespan in which we were able to go from 0 to MVP. On Saturday 9/28, after going to the gym and taking care of morning routines, we started development. 8 hours later, we had the initial sparks of a minimum viable product. 8 hours after that and we were buying the domain name, uploading the code onto Replit to be hosted, and setting up an SSL certificate on Cloudflare. It surprised me how it only takes one day of being locked in to make a project go from ideas to repositories.

Another thing we are proud of is the look of the frontend. With the animation in the background adding a subtle touch of movement and sleekness, along with a nice color scheme, our frontend design looked better than we originally anticipated.

What we learned

We learned how important a time constraint is, and how it can actually be beneficial to ensuring project progress and completion. Under the circumstances of the hackathon, we learned to disregard slight imperfections in place of changes that would actually add substance and be more profound. One rule slightly related is the 80/20 rule, where 80% of the work is done in 20% of the time and vice versa. Instead of trying to make that 20% perfect and waste 80% of our time, we learned to get the groundworks in and get a working prototype before focusing on the details.

What's next for BuyBetter

Although BuyBetter currently works phenomenally, the backend will have some performance issues once our userbase grows. Currently, we have plans to optimize the backend by deleting receipts from cache once the completed receipt is passed to the frontend. The frontend pinging method could also be changed to include exponential backoff, reducing the load on the server from 2 requests every second. We also want to include other information from the receipt including total costs, date purchased, and store name/location. This extra information would be helpful in making alternative purchase suggestions that are tailored more closely to the user.