Managing finances and spending can be a messy problem when you have hundreds of receipts from purchases throughout the year. In certain cases, finance and money management is the only thing between some people and achieving their goals. Most people just end up throwing away their receipts, but we wanted to create an easy and more responsible way to keep track of your spending, regardless of if you use cash or card.

What it does

The user can take a photo of a receipt and have it automatically record the total amount of money they spent on the purchase.

How we built it

We used Google Vision API with the camera so that users would see boxes over the text that the phone detected. This would let the users be sure that they were getting a reliable image. Once the user tapped the screen, a photo would be taken that would be uploaded to The OCR component of Microsoft's Computer Vision API.

Challenges we ran into

This was the first time programming in Android for every member of our group, so almost everything was new to us. We had a few hangups with the permission settings for taking a photo with the app and then saving it. The mentors were a great help in getting around these problems. We also had trouble getting useful data from the OCR API, because ultimately we are limited by the state of OCR research. We got around this partially by being adding other options for words which could be detected.

Accomplishments that we're proud of

Diving into OCR as well as Android for the first time and coming out successfully.

What we learned

We learned a lot about building an android application as well as utilizing machine learning API's.

What's next for Receipt Itemizer

Detecting other components of the receipt such as the individual items purchased. We'd also like to use data visualization to record spending over time.

Built With

  • google-vision
  • microsoft-ocr
Share this project: