Recei-Bot

Scans reciepts to collect and record financial expenditures.

Comment

Inspiration

  • Receipts are clunky, disorganized, and disconnected. We need a way to store receipt data into a common place that is not our wallet.
  • Receipts in union can reveal overspending, which can be hard to pinpoint by yourself.
  • Most services for receipt scanning are geared towards businesses and have paid subscriptions.

What it does

  • Scans important reciept such as purchase price, date, and vendor.
  • Spending can be plotted over time and grouped by vendor.

How we built it

  • EDA with python data science kit (numpy, pandas, scikit-learn, etc) and OpenCV to characterize receipt image data and expose problematic factors
  • Used Tesseract OCR to read text from a receipt and categorized it using delimiters (such as "date" and "[month]" for the date and "$", "Total", and "Amount" for the price)

Challenges we ran into

  • Getting exact company names and addresses is very hard
    • Most people shop at the same places over and over
    • A simple autofill / autocorrect feature can be implemented by storing existing company names and addresses in a trie.
    • Addresses can be verified using the Address Validation API by Google Maps

Accomplishments that we're proud of

  • Wrote an depth EDA that explores image quality metrics
  • Built a somewhat functioning model using Tesseract OCR

What we learned

  • Use tools that already exist --> Apple Vision OCR, Microsoft Vision OCR, Google Maps
  • Learn React.js, Kotlin, or Swift so I can actually deploy our program to a website or mobile app

What's next for Reci-Bot

  • Better image preprocessing using problems found in EDA
    • This can be partially done by prompting the user to crop their photo
  • Refine the OCR model or use an existing one
  • Build or find a model that can categorize purchases into groups such as Food, Groceries, Entertainment
  • Implement a SQL database to map user IDs to receipt data.
    • This can also be stored locally, but having server-side data will allow for data transfer across platforms and/or allow us to collect data
  • Deploy the service to a mobile app or website

Built With

Share this project:

Updates