That's Bullish!

Inspiration

GME to the moon. As avid stock investors ourselves, we realize the large potential in how mass views on the internet can sway the stock market, sometimes even moreso than hedgefunds, and thus sought to capitalize on it.

What it does

The project scrapes reddit for the hottest posts , and then filters through the comments and rank the most talked about stocks.

How we built it

We built the project on Microsoft Visual Studio Code using Python and Selenium to web-scrape for the urls of each post, and then used a python reddit api wrapper (PRAW) to filter through the comments to identify top talked about stocks. The scraper uploads data to a Firebase database. The front end, which is implemented with React, retrieves the data from the Firebase database and displays it to the user.

Challenges we ran into

One major challenge we ran into was when the chrome web driver opened while scraping, a pop-up would appear from google chrome asking to allow notifications for the website, that would hinder the bot from locating the xpath to gather the links of each post. Since the webdriver was new each time, we could not manually disable the pop-up ourselves once. We solved this by writing code that dove into the settings of the chrome webdriver and set the notifications off by default. One other challenge was whilst gathering the links of each post sequentially, an occasional ad disguised as a post with no comments relating to the stock market would be in our database. To solve this in our scraping program we added an if statement noticing that only legitimate posts had the keyword "comments" in it, and thus if the url did not contain it, should not be in the database to be used by the API. Another challenge we encountered was while filtering through comments, finding words or phrase acronyms that matched with company abbreviations, such as LMAO or ALL. To solve, this we added a manual filter to filter out tickers that are more likely to be normal English words.

Accomplishments that we're proud of

We're very proud of the comment searching tool that ranks the hottest talked about stocks on the subreddits, as this is information we usually gather ourselves.

What we learned

We learned so much about scraping websites, using bots, and gathering information using api, which would likely be useful in automating other sorts of resource information gathering.