TipTracker

Inspiration

As high school students with experience in service industry jobs like baristas, servers, and retail workers, we understand firsthand the critical importance of tips in these roles. Countless studies show that tips account for over 50% of earnings for these jobs. As a barista, I have heard concerning stories from colleagues about past experiences where their hard-earned tips were withheld. This is what first inspired us to create our project, as we wanted to create a platform that gives tips the attention they deserve. Using this platform, employees in restaurant, hospitality, retail, transportation, and all other service industries will not only be able to flawlessly and effortlessly track their tips, but also receive valuable insights from the data analysis that is seamlessly integrated into the app. This platform ensures that service workers can accurately monitor their earnings and leverage data-driven insights to optimize their income potential.

What it does

TipTracker is an application created to help people who have jobs in the service industry track their tips to maximize their earnings. Once the user has logged into the platform, the user can choose to upload their tip information into the app by either manually entering it or uploading their receipts from customers, which can be scanned in mass to avoid having to manually type out the tips from each receipt. Additionally, there are visual features like a graph that shows the amount of tips you have earned over time, showing the amount of tips earned on each data and time. The platform provides advanced analytics like AI insights that give information like what time the employee earns the most tips, what clothes the employee tends to wear when earning the maximum tips, and more. There is also a feature of a tip log, that has a record of every tip earned. The platform has so many more amazing features, so please check it out on your own time!

How we built it

Building Tipsterz involved a combination of AI-driven OCR technology, structured data analysis, and a streamlined UI/UX design to create a seamless tip-tracking experience for gig workers. The foundation of our AI strategy was the implementation of Tesseract OCR, an open-source optical character recognition engine, to extract tip amounts from scanned receipts. We optimized the accuracy of OCR by applying image preprocessing techniques, ensuring that Tesseract could accurately recognize numerical values from printed and handwritten text. The extracted tip data was then parsed, formatted, and stored in our backend for further processing and analysis.

On the data analysis side, we structured a system that tracks tip history over time, enabling users to gain insights into their earnings patterns. By storing tip data in a database, we built an analytical framework to evaluate trends such as peak earning periods, seasonal fluctuations, and tip distribution across different payment methods. This laid the groundwork for advanced AI-driven features, such as predictive analytics that suggest optimal working hours based on past earnings and external industry trends. The system also supports comparative insights, allowing users to see how their tips compare to industry averages.

The UI/UX of Tipsterz was designed with React and Tailwind CSS, ensuring a clean and responsive interface. We utilized Vite for a fast and optimized development environment, improving the overall performance of the application. The form system, powered by react-hook-form and Zod validation, ensures smooth user interactions, while API requests handle seamless data submission and retrieval. The OCR scanning feature was integrated directly into the interface, allowing users to upload receipts and instantly view extracted tip amounts, minimizing manual input.

For backend development, we structured API endpoints to handle tip submissions, retrieval, and analytics processing. Our system enforces strict data validation, ensuring that all submitted tips follow the correct format, including date parsing to prevent errors. We leveraged caching strategies and query optimizations to make data retrieval efficient, allowing for real-time updates on earnings trends and statistics.

Challenges we ran into

A major challenge we ran into was creating the code for the receipt scanner. We began working on our project by training a CRNN model to read the receipts and gather the information we needed, like tip amount and date of transaction. Our accuracy rate was initially lower than expected, so we continued to work on that for a few hours. We eventually solved this issue by building a pipeline that leveraged Tesseract OCR, which made our code much more efficient.

Accomplishments that we're proud of

This was both of our first hackathons, so we are very proud of ourselves for working hard to address an issue that is incredibly relevant. Additionally, we were able to effectively implementing an AI model into our project, which was something we were hesitant and unsure about going into this hackathon.

What we learned

Building TipTracker taught us a lot about AI-driven OCR, data analysis, and user experience. Using Tesseract, we quickly realized that raw OCR outputs weren’t always clean, so we experimented with image preprocessing to improve accuracy. Handling messy receipt formats also pushed us to refine parsing logic and enforce strict data validation with Zod to avoid backend errors.

On the analytics side, we learned the importance of data normalization to make tip trends meaningful. Identifying peak earning times showed us the potential for AI-driven insights, and optimizing queries improved performance. For UI/UX, we focused on keeping things fast and intuitive, making receipt scanning effortless and ensuring mobile usability.

What's next for TipTracker

Tip pooling is a very common practice in the restaurant industry. Our next step is to expand our platform to have a space for restaurants to make communities for their employees where employees can track their tips and then fairly split them amongst themselves. Our next step after that is to connect TipTracker to platforms like Zelle and Venmo, allowing employees to cash in their tips themselves instead of having to wait for payday.