We found that there was a lack of data available to the general consumer that they could easily understand. When we saw the opportunity to solve this problem through the products offered by the sponsors at PennApps such as Goldman Sachs Marquee and Twilio APIs, we jumped on the opportunity.

What it does

Fin has the goal of giving financial data and advice in the most intuitive way possible, through the use of a simple chatbot interface, and the option to view data in AR. By communicating with Fin, you can perform the following tasks:

  • Buy/sell stocks
  • View price history of stock in AR
  • Get stock recommendations
  • Find similar stocks and view them in an AR constellation

How we built it

The project is divided into two main sections, the UI and the backend. The UI is built using react-native, and uses ReactViro as an AR library. All commands are passed through edas chats with our bot, which is run through Twilio Programmable Chat, which connects to a Twilio Autopilot. This Twilio autopilot then calls various endpoints on the backend so that it can retrieve data and reply to the user.

The backend is built on a server-less architecture using AWS Lambda. This retrieves the data and performs calculations to answer user questions. We use IEX for retrieving stock prices, Alpaca for paper trading, Marquee for portfolio analysis and stock recommendation. Finally, we can find similar stocks by using the BERT deep learning technique on descriptions of each company in the S&P500.

Challenges we ran into

  • Inconsistent positioning and rotation of various objects within the Viro library
  • The Twilio API being very selective about inputs received
  • Lack of well packaged financial data from a single source
  • Getting BERT working

Accomplishments that we're proud of

  • Using cutting edge deep learning techniques to classify similar stocks (BERT NLP)
  • First time using any kind of AR
  • First time using BERT
  • Creating an AR interface that is intuitive to use

What we learned

  • How to use ReactViro to build AR apps
  • How to construct fixed length vector embeddings of variable length using BERT
  • How to use AWS lambda

What's next for Fin

Next we want to add more commands, improve the user experience, and provide a bit of a guided approach so that someone who has never traded before can get some good advice and play around a bit to understand the market better.

Built With

Share this project: