Banana Split

Login Screen
Image Input Screen
Receipt Session Screen
Extraneous keypoint removal example
Final receipt contour example
Result of perspective transformation example
Example text output
Image of the database tracking all the data
Photo of Venmo request screen

Inspiration

When we get groceries with our roommates or buy some food for a large group of people, there is usually a long, confusing receipt. People end up spending more or less then what they ate/used, and it can take a while to calculate the total sum. What if there was a more logical way to organize splitting a receipt so that each person could pick their individual items on their own device. Or what if there was a more granular method of splitting up items so you could avoid going halfsies on a dozen eggs when you only want one.

What it does

Instead of going through the receipt person by person, item by item, to calculate each person's contribution, whoever paid - we'll call them the host - can simply take a picture of their receipt and let Banana Split take care of the processing. Our app applies a load of computer vision algorithms to process, transform, and perform OCR on the receipt, parsing each item purchased and its associated price (and grocery store food group!). The host can then quickly send a link to whoever needs to pay up in a group chat, and it's hands off from there. Each person then individually logs in with their Venmo account and is presented with a list of items to select from. They can choose to pay for as much of a particular item as they think they'll use. Once everyone has completed their shopping carts, the host receives a confirmation text and, upon confirmation, Banana Split uses Venmo to send a request to each person for the amount they owe.

How we built it

The frontend was built using React. The backend was built using a light Flask server with endpoints to the Twilio API, Venmo API, our computer vision algorithms, and our Mongo DB which stores multiple sessions. Each session corresponds to a different receipt and its users, items, and prices.

Challenges we ran into

We got our IP banned from the Venmo API for making too many requests while testing. Turns out there are safeguards to mass sending Venmo requests to random strangers.

Accomplishments that we're proud of

We're pretty proud of the clean UI/UX and how we got all the different pieces of technology to cooperate . Our application also ended up being pretty robust and supports a lot of interactions between multiple users in a session.

What we learned

Computer vision and full stack development.

What's next for Banana Split

Get it deployed so that we can use it routinely :)

Built With

Submitted to

VTHacks 8
- Winner (1st overall) The Everest Award
- Winner Capital One | Best Financial Hack

Created by

I worked on processing input images and parsing information from receipts. I used opencv to do things like extract keypoints and perform perspective transformations.

Sam Schoedel
I worked on the UI/UX of the application along with the overall architecture.

Nathan Moeliono
I worked on the MongoDB database and back end. It was my first time working with Databases, so I went with Python and Pymongo to handle the updates and queries.

Justin Vita