CryptoSensAI

System architecture

Inspiration

We are a group of Software Developers and Artificial Intellegence Specialists that like to invest in cryptocurrency. Cryptocurrency investment is highly volatile and influence by events and personalities. We were looking for a powerful tool that allowed us to contextualize language and create a semantic network to have a objective control over the decision making process. ExpertAI allowed us to do so, integrating its cognities capabilities within this problem.

What it does

CryptoSensAI is an intelligent system created to help tracking all these events to aid you and your investment decisions. CryptoSensAI does so by automatically:

Crawling specific Internet forums and certain celebrities social notworks to extract what is being said about different cryptocurrencies.
Analyzing the extracted texts using the ExpertAI API to get the overall sentiment and the involved entities, topics and events.
Presenting the information to the user on an easy-to-read and interactive dashboard.

How we built it

Thanks to the flexibility and highly documented ExpertAI API we followed a rapid-prototypying strategy, wich allowed us to obtain reliable results in a short time with not much collected data. The solution consists of three main layers:

Ingest: by using crawlers and specific APIs we extract text data from social networks and financial news forums.
Data processing: through an intensive use of the ExpertAI API we extract sentiments, topics, entities and the impactful relationships between them.
Discovery: this layer transforms the data into a collection of dynamic graphs that show the top trends and metrics related to the text and sentiment analysis. This layer was designed to help the user on its investment process, allowing him to customize his/her own graphs and visualitation tools if he/she so desires. This layer uses the power of Elastic and Kibana to accomplish its goals.

Challenges we ran into

We run into some different types of problems:

Crawling: we run into some issues while crawling, due to cookies and other website interactable elements.
Data quality: heterogeneity with the extracted texts, the texts are rather different from each other. Different lengths, different ways of answering messages, use of locally specific terms (Reddit slang, Yahoo Finance references...), etc.
Polarity and criticality: it was hard to contextualize and read into these text characteristics in the cryptocurrency world in an accurate way.

Accomplishments that we're proud of

We are rather proud of all the overcome challenges. We really appreciate all the knowledge we have gained during the development of these project and specially the intersubjective knowledge about cryptocurrency that this cognitive tool is capable of extracting.

What we learned

Outside of a technical level we have learned a lot about the state-of-the-art of cryptocurrency investment and cryptocurrency investment applications. On a technical level we have also learned a lot about the ExpertAI API, Kibana and the capabilites of Natural Language Understanding.

What's next for CryptoSensAI

Right now the project is in a Proof-Of-Concept state. We hope this prototype will be succesful so we can keep building on top of it, adding more webpages and social networks to crawl, and exploiting even more text characteristics. The only limit is our imagination!