Having recently joined the world of stock trading, I wanted an edge when selecting which stocks to choose and which ones to dump.

What it does

Using expertai's sentiment analysis tool, I quantified SeekingAlpha's May 2021's news stories and then compared that company's stock price movement for the next 1, 3 and 5 days.

How I built it

Using PHP and MySQL, I first obtained all of SeekingAlpha's news stories for the month of May 2021 via scraping. I filtered out the stories that contained more than one ticker mentioned, in order to only analyze stories that focused on one particular stock. I also limited the stories from May 1st to May 28th, keeping a tidy 4-week collection of items, totaling 5,836 articles.

I then ran all of the news stories through expertai's sentiment analysis tool and obtained each one's sentiment score, which I added to the article in the database.

The next step was obtaining each stock's price movement for the period from May 3rd to June 7th (to properly measure the fifth trading day for stories that ran on the 28th of May, taking into account that May 31st was a holiday). I used as a source and downloaded the NYSE and NASDAQ databases for that time period.

The final step was to create the visual component on the front page that lets the user select a "trending" ticker (in this case, a ticker with many stories) or manually enter one, and see how it behaved in that time period, highlighting the positive and negative stories associated with it and its results (gain/loss percentage) for the next day, 3 days and 5 days. I also added a stats page that lets the user see the results of the project from a "30,000 foot" perspective, taking all the stories and their sentiment scores into account.

Challenges Iran into

The biggest challenge was not integrating expertai's API (that was the easiest part) nor was it obtaining the stock market data - it was scraping the news stories and making sure that the stories focused only on one company. Mining "good" data is the key for a tool like this to work.

Accomplishments that I am proud of

That my hypothesis was correct - negative stories result in a short-term loss (on the average, 80% of negative stories equated to a drop in stock price for the next day) and positive stories result in both a short-term and long-term gain (79% of all positive stories resulted in either a 1-day, 3-day or 5-day gain).

What I learned

I learned a lot about the real-world application of sentiment analysis and that an article, whether positive or negative, can assist in the decision-making process of whether or not to execute a stock trade.

What's next for Stock Sentiment Analysis

I would love to be able to grab news stories from several other sources and get their sentiment score. A daily cron job to gather up-to-date stock market data would also be great, as well as a predictor tool, which could take into account up-to-the-minute news stories and run multiple simulations to see where the stock price may end up in the coming days.

Share this project: