API: Vision OCR -> implement a mobile app Having a bunch of receipts? Want to have clear and accurate data on your spending? “Recapture” is your best friend. All these data from taking a picture.

Features: Summary of statistics and spending Easy and convenient and fast -> Users have options to correct or fill in incorrect interpretation -> machine learn from these. User side: Beautiful. Easy fast and convenient to post all your budgets and have a good summary of what to spend. Just take a picture to have all the data stored in a useful fact sheet. See how much you spend on something. Can manual input.

  1. Summary of spending, budgeting a. You know how much you spend on average per month for each category b. What to cut what to improve, budget goal?
  2. Everything categorized -> total amount spent per month, per week, per year. Graphs of trends in spending -> UI should be simple and give more information when clicked on. Swipe right swipe left for more stuff.
  3. User correction -> if auto recognition guesses it wrong, then the user can auto correct and the app will learn how to do it better next time. a. Implements machine learning. Uses MS API tag and improve tag based on what the user corrects b. Splits images into small parts and targeted machine learning to recognize parts that are titles
  4. More info on subcategory a. What did you buy the most b. Where did you buy at c. How much money you buy

Example: Interesting usage… Does gas from different station affect your car’s mileage? For every gas receipt, you obtain day, cost, #of gallons, and which gas station from the receipt. The user easily inputs how many miles since the last fill. Then special gas category will tell them average mileage per station. Maybe it doesn’t make a difference but maybe it does.

Business side:
One of the biggest subject in marketing and business now is obtaining meaningful data for consumer analytics… Google ads, Amazon, Facebook, and MS and many other companies are trying to obtain meaningful data about consumers behaviors and such. Morality issue: Anonymity and such. Only use for marketing and advertisement purposes. Same as facebook and google and Amazon on obtaining information about their customers. Process and predict and learn about the economy.


  • Interpret API output

- Guessing image correctly.

Points to improve

  • Right now the vision API recognition is not perfect. Blurry text or crumbled up receipts might not work too well.
  • Further categorization of data -> subcategories
  • Can manually input data
  • Really good machine learning -> strong image recognition and correct tagging. Learn from multiple users.
  • Can have additional features such have text search for receipt image (storing receipt image in a cloud or local somewhere).
  • Keyword rank -> user interest
  • Faster Potential for machine learning:
  • Tagging system saves a database of common items and tags
  • More accurate tagging

Built With

Share this project: