Inspiration
As a crypto enthusiast and hobbyist trader i wanted to keep up with the latest news. Twitter is indeed the right place for that, many of the interesting things appear first there. The problem I faced was that it is very hard to get through all the content to find the important tweets. So I decided to create a filtering system that will do that job for me.
What it does
Crypto Portal is a content aggregator with some filtering mechanics. It scans different sources of news, tracks certain accounts, measures altcoins deviation from bitcoin, creates keyword trends and tries to detect important tweets.
How I built it
The application has the following modules:
- Python content generator and filter. This module contains: tweepy filtered stream, tweets and accounts lookup, selenium web scrapers, API of crypto exchange, scripts that filter the content, create averages, manage the database and create archives of data.
- Online database (Pymongo) with REST API
- Android (Java) fragment based application. It uses Firebase services to receive notifications, Retrofit for REST Client and MPAndroidChart to visualize chart data.
Challenges I ran into
The biggest challenge I faced was and still is to create an algorithm that evaluates the importance of a post without human intervention.
Accomplishments that I am proud of
I am proud of creating a complex system to gather and display content and fully takes care of itself. Probably the most interesting feature is the model that detects exceptional tweets engagement.
What I learned
I learned that data mining and machine learning techniques can provide excellent solutions to many problems.
What's next for Crypto Portal
I am currently working on the following features:
- Algorithm that finds if a certain piece of news has appeared anywhere else before.
- Model that tries to detect if a particular status is causing a movement in currency price.
- Twitter List model, which given certain accounts finds similar ones.
- To perfect the engagement score by taking in account correlation between likes, replays, retweets and anomalies in those.
Log in or sign up for Devpost to join the conversation.