Leveraging the cutting edge data science to make wise investment has always been an interesting topic. In this application, the data that attracts our attention is news. As we believe, the more people think a company has a promising future, the more likely the market capitalization is going to grow. Therefore, we came up with an idea to apply sentiment analysis to news using Natural Language Processing to predict the stock trends, and thus to provide investment recommendation.
What it does
- This application predicts stock trends so that investors can be aware of potential changes to their target stocks.
- Our project analyzed the sentiment of news on all listed companies through data mining in Natural Language Processing.
How we built it
- Implemented python web crawler to fetch news on target companies
- Used IBM Bluemix Watson APIs to analyze news.
- Run the server on IBM Bluemix Cloud Foundry.
- For frond end, used nvd3.js to display trends of stocks and sentiment value generated by Bluemix.
- For back end, used Flask and sqlite.
Challenges we ran into
- Hard to find source for mining valuable data.
- Hard to curate data for NLP analysis.
- In developing the front end, we need to design a graph view to best display the result of stock and news sentimental trends. We tried different tools and different graphs.
- For backend, we analyzed and tested APIs of different providers to choose the fastest and accurate source analyzing tool.
- We also spent much time to analyze correlation between companies' positive/negative popularity on news and stock trends. Sometimes it cost much time to mining the news data since there are massive information in data set.
Accomplishments that we're proud of
- From previous data we can prove the prediction is highly reliable.
- Vividly displaying stock trend and investment recommendation
What we learned
- Expose ourselves to technologies including IBM Bluemix, nvd3.js, Flask and etc.
- Cooperated to develop a product
What's next for NeXTock
- Improve model to make more reliable long-term prediction.