The problem

  • 31% of adults globally are unbanked - meaning 2.4 billion aren't registered with a banking institution and generally pay using cash or money orders.
  • Young adults and women, especially from less fortunate households in developing countries are more likely to be unbanked. Among those living below the UN poverty line of $2 per day, 77% do not have a formal bank account.
  • Given that 80% of a developing countries live in rural areas, it is not financially viable to service them through traditional banks, hence a tech-based service is required.
  • Although 84% of this demographic owns a mobile phone, most of them are not smartphones - lacking modern app and internet capabilities.
  • Since the traditional banking system was not designed for less fortunate people in developing countries, we dedicated this weekend to build a new solution.

The solution

  • Our solution Pennyroyal, is an SMS mobile banking system that allows clients to bank remotely without having to meet with an agent. Instead of meeting with an agent, clients can perform formal identification and begin banking using their phone.
  • This technology was inspired by M-Pesa, the first SMS mobile banking service in Kenya.
  • The goal of Pennyroyal is to improve accessibility, security, and financial enablement for the unbanked population.
  • We've made banking more accessible by making all important features available through a user-friendly SMS interaction-based interface.
  • Clients can register an account using their phone number, deposit cheques by taking a photo, send money to others, and check their balance, all through SMS text messaging.
  • Connected to Ethereum blockchain to protect against data loss in the event of a database crash. This ensures that all our users' account data and money are secure.
  • We care about security. We implemented a PIN based authentication system to enable cash transfers via an automated authentication call.
  • We mitigate risk of bank accounts opened by fake users by having users take a selfie and a picture of their picture ID and sending it to us for review. In our backend we use an MTCNN algorithm to extract faces from both images and compare them with Keras API's VGG face model boasting a 96% accuracy.

How we built it

  • We utilized Twilio Studio Flow to handle SMS messaging and conversation flows. We chose to use their technology because of the simple conversation state management tools.
  • Accessed Ethereum blockchain with Infura API.
  • Built PIN based authentication with Twilio API.
  • Registration facial analysis done with VGG model Keras API.
  • Analyzed cheque deposits with TesseractJS, an Optical Character Recognition (OCR) library.
  • Our production hosting includes Twilio Studio Flow for handing SMS messaging, Heroku for hosting our Node API server, and Infura for connecting to Ethereum blockchain.

Challenges we ran into

  • Speed — Our tech can be slow at times when writing to Ethereum and our network traffic isn't very efficient since we need Twilio to make requests to our backend which subsequently makes requests to Ethereum and that passes it back down the chain.
  • Extracting data from cheques for deposits. Extracting info from handwriting is difficult. We didn't have time to train a ML model that could interpret handwriting. All we could do was extract banking info.
  • We initially chose to work with Twilio because we thought managing conversations with their platform would be easy. That turned out to be false. Managing conversation state was easy, but to process image data from SMS we had to write very convoluted and redundant code.

Accomplishments that we're proud of

Honestly this was the first time we pushed ourselves this hard at a hackathon. We decided on an ambitious idea that utilizes technology completely outside of our comfort zones, and we worked long hours to get our service up and running smoothly. When one of us got stuck, we helped each other out and worked together to make it work.

  • Working with ML for the first time
  • Processing cheques with ML
  • Extracting faces from picture ids and selfies and comparing them with ML
  • Using Python together with Node
  • Developing good user experience chat flow
  • CRUD data on blockchain
  • Broke down project with Notion, developed under agile methodology, and collaborated on repository with Git flow.

What we learned

  • The severity of the unbanked population and the consequences to the economy
  • How to use blockchain to secure data and prevent data loss
  • Coding in Solidity, deploying smart contracts, and how Ethereum blockchain works.
  • Implementing secure authentication and bank cheque analysis with ML
  • How to use Keras API, OpenCV, MTCNN
  • Twilio studio flow
  • Receiving and processing images with Twilio

What's next for Pennyroyal

  • Currently, Twilio sends HTTP requests to an Express server which uses the Web3 library to talk to the blockchain. This is redundant and we will be integrating Twilio's API into our server instead of using their platform.
  • Ethereum blockchain is slow, and in the future we would look into using a different blockchain.
  • Cover edge cases — because this was a hackathon, we spent time developing the core functionality, and didn't cover situations such as cancelling a command.
  • Our original plan was to implement the option to request loans and generate credit scores with data from your phone, but didn't have time to train a model and permissions to extract phone data through phone number lookup
Share this project: