final image 2
For people on the go it is both impractical and a tremendous hassle to maintain records of transactions made. Budget Bucket hopes to provide a helpful alternative for people who need to record their finances, but cannot allocate the time to fill in forms and boxes.
What it does
Instead of filling in boxes, scrolling through options, and saving pictures of checks, Budget Bucket allows the creation of quick and automated financial records with only the click of a camera. All the necessary information is automatically extracted from the receipt and stored on the cloud. Users can then easily view and retrieve it, as well as see important summaries at a moment's glance. With the help of automatic classification of these transactions, as well as saved images of transactions, financial records are easy to find and extract.
How we built it
Budget Bucket is constituted of a number of parts. An image is uploaded through an app built with React Native, and then is processed by OpenCV on a Google Compute Engine server to be brought to a machine-readable version, which is converted to text by Google Cloud Platform tools, which is then fed to IBM Watson in order to find details of the transaction made, which is finally stored in a MongoDB database. This information can be accessed through the app and a website built with Bootstrap and Flask.
Challenges we ran into
The main challenge we ran into was the networking of the mobile app to the server. At first we had major issues with this, but in the end we managed to solve it by shifting the part of our stack that addressed that particular part. Another major challenge we ran into was the detection of checks not working if the check was warped in any way, however we managed to fix this by lowering our overall accuracy for greater range.
Accomplishments that we're proud of
Using OpenCV to capture the contents of the transaction regardless of orientation, size, and color. Employing the integration of Watson technologies from IBM and Google cloud services integrate machine learning into our application
What we learned
We increased our knowledge of machine learning and image processing, especially OpenCV, IBM Watson, and Google Cloud Services. We learned more about React Native and its extensive use in mobile application development. We learned how to pass Flask into a reverse proxy running on nginx.
What's next for Budget Bucket
We plan to add user authentication and the ability to add more information about a transaction than what is already understood from a picture.