Accessible mobile check depositing

Inspiration

After brainstorming, our team all knew that we wanted to make an app to help individuals not often represented in technology. Technology can sometimes be hard to understand and to work with, especially for the visually impaired. Depositing checks with a mobile app can be difficult for the visually impaired because it is not always clear where you need to move the phone relative to the check for the image to be taken.

What it does

Our accessible app will give the user cues as to which direction they should move the phone in order to capture the image, handing power to visually impaired users who previously may have needed assistance with this technology. Our app also talks to the user, helping them navigate their banking with ease, and using swipes back and forth to navigate between screens, for a no fuss, easy to read presentation of their banking information.

How we built it

We used audio and image help the client place the image to the center of the detection box We used opencv's image transformation tools to flip the image (if necessary) and turn it into a greyscale image We used opencv, python, and yolov3 to detect the total amount We need to somehow crop the detected area and save it to another image We need to detect Unique characters from that cropped image (code present, not modified) We need to use MNIST dataset and Caffe to recognize particular characters (source present, not done)

Challenges we ran into

Detecting the check is all or nothing. We needed to find a way to detect the check when it is only partially available Training the custom yolov3 model from a CPU computer. Our laptop doesn't have GPU's built into it...so it ran out of memory before we finished training

What's next for Enhanced check depositing

Parse the payment digits. Integrate PayPal/Braintree/Venmo to help users hear and interact with their account balance.

Built With

• android-studio
• darknet
• gcp
• opencv
• python
• yolov3