Given rising student debt levels and income inequality, we wanted to create an app that will help students learn about personal finance and gain insights into their spending habits.
What it does
Our application consists of two parts: an OCR receipt scanning service and a statistics engine. Using multiple ML models, we determine the content of a receipt and a line-by-line review of purchases and assign them to a general category. Using these categories, we then perform statistical analysis of purchasing habits and provide a comparison of your spending habits to those in similar situations.
How we built it
Our backend is built using Flask with MongoDB as the database of choice. The front-end was constructed using Nuxt.js. ML and OCR capabilities were provided by Microsoft Azure. Data was provided by the TD Da Vinci API.
Challenges we ran into
Recognizing receipts was extremely challenging. Not only do layouts differ widely between merchants, understanding product names was also a challenge. We ended up performing image recognition on the first image search result of the product and getting a general category from that. Working with a dataset of millions of entry was also a challenge for us because we were not used to processing such loads efficiently.
Accomplishments that we're proud of
Successfully recognizing different receipt formats and deriving product categories from a product name on a receipt. Changing the color of the card to dynamically match purchasing habits like the Apple Card was also very cool.
What we learned
We learned a lot about working with large datasets and building custom ML models to perform tasks.
What's next for Accountable - A personal finance app
Since the TD API is only meant for demo purposes, we just aim to have this as a proof-of-concept.