Inspiration

Sentiment data are noisy. We want to find some way to identify those have better predictive power. Individual stock returns and broad index returns have been shown to not be serially correlated. So we grouped them to form a higher correlation.

What it does

We chose 100 stocks with highest market capitalisation and grouped them into four groups based on correlation. We did this by implementing the Louvain community detection algorithm, which is known for its speed and minimal storage requirement. We then performed a time series analysis on each group and used this model as a baseline. We then added sentiment and macroeconomic data to investigate how this would improve the stock price prediction power to ultimately be used in a trading strategy (not part of this). The method would provide a way to compare the predictive power of different sentiment data sets, for example, data sets of all news headlines, data sets of all financial news headlines, twitter feeds etc. It can also be used to compare the predictive power of macroeconomic data. By doing this, we can reduce the computation cost of modeling using all sentiment data/macroeconimic data.

How we built it

Jupyter notebook developed during Algothon2019

Challenges we ran into

Time taken to run models within the 24 hours of the event. Lots of different ideas we wanted to try but did not manage because of time constraints. Consistency of predictive power across datasets.

Accomplishments that we're proud of

What we learned

What's next for SentiMeter

It would be interesting to test different data set and pick out ones that help reduce the cost function the most. We can then train a model based on these features and transform it into a trading strategy.