We set out on a mission to unify multiple datastreams in an effort to gain a competitive edge in the world of crypto trading. Our advantage over other programs is Reinforcement Learning (RL), an area of Machine Learning. RL agents have the ability to learn continually and dynamically mimicking what a real-life trader does. An RL agent has access to past price data, trading indicators and news sentiment analysis just like a trader, they then can make decisions in this environment: buy, hold, sell and learn from their mistakes creating the best possible trading strategy. This was our mission, and not to spoil the ending but we succeeded.
What it does
Our trading agent is a new take on the traditional trading bots leveraging the power of NLP, Reinforcement learning and multiple streams of data. See the image for an overview of how all the parts of the Uniter Agent fits together.
It takes in price data, news sentiment data as well as trading technical indicators and it outputs an action: buy, hold or sell for a specific crypto currency. We then run this through a testing period to analyse the performance of the agent.
How we built it
Binance API for all crypto trading data Technical trading indicators used: macd, boll_ub, boll_lb, rsi_30, dx_30, close_30_sma, close_60_sma
We leveraged state of the art pretrained language models (BERT) to identify sentiment from tweets and articles giving an insight into attitudes towards a particular coin throughout the year. The crypto market is known to be heavily speculative which made this the best market for us to deploy this technology. These sentiments were then aggregated over a 4 hour period.
Challenges we ran into
Trained and tested 3 models: Soft actor critic, DDPG and Proximal policy optimisation (PPO) With a range of hyper parameters: trading interval (5min, 1hour,4hour,8hour,1day) Training took between 30 and 45 minutes per test which meant we had to focus on only experiments we needed to run to learn more about how the agent and environment works.
PPO was the best algorithm for this and it is the one in the final agent.
The Twitter API was extremely hard to work with and limited us on the number of tweets that we could retrieve per hour and the search criterias that we could apply.There were some open source libraries but they were outdated and no longer worked, so it was a challenge to find one that fits our needs. However, after a lot of searching, we came across one library that had been built to scrape on a much larger scale and this allowed us to collect enough data to facilitate learning.
Another issue was that a lot of tweets were noisy. They would contain coin keywords but the main content of the tweet would be completely unrelated. To tackle this, we added extra cryptocurrency keywords to our search parameters to ensure that the tweets were on topic.
Scraping data from Google news also proved to be a challenge. Google news does not have an API meaning we had to find an open source library that directly scraped search results. Then using search filters ‘before’ and ‘after’ in google, news articles from precise dates could be scraped. We decided to only use the news headline and description of the article in analysis to reduce noise in results, increasing confidence in sentiment.
Accomplishments that we're proud of
Scraped over a year's worth of data from Twitter and Google News feeds regarding cryptocurrencies. Built and backtested a RL powered crypto trading agent in under 24 hours. Achieving 45% Profit!! HODL (Buy / Hold strategy) - benchmark
Statistic Value Annual return 0.070374 Cumulative returns 0.409945 Annual volatility 0.296061 Sharpe ratio 0.377606 Calmar ratio 0.178156 Stability 0.747045 Max drawdown -0.395016 Omega ratio 1.068705 Sortino ratio 0.554558 Tail ratio 1.086825 Daily value at risk -0.036857
Our Agent - Uniter Bot
Annual return 0.077438 Cumulative returns 0.455854 Annual volatility 0.285704 Sharpe ratio 0.403711 Calmar ratio 0.227455 Stability 0.762475 Max drawdown -0.340454 Omega ratio 1.074838 Sortino ratio 0.598638 Tail ratio 1.107681 Daily value at risk -0.035538
What we learned
How to scrape Twitter given the recent removal of the Twitter API for non academic sources to get many thousands of tweets. How to scrape Google news page results to get thousands of articles Implementation of RL using sentiment data and trading metrics on a highly volatile crypto market with really good return results! How to preprocess textual data in order to extract the core content How to leverage large pretrained transformer models in order to understand sentiment How to tune hyperparameters for the stock/crypto trading environments - completely different the openai gym atari environments
What's next for Uniter Bot - RL crypto trading bot
- Integrate with the Twitter API directly to access the most recent real-time tweets for use when making decisions.
- Collect more news and Twitter data over a longer time period to give the agent more to learn
- Integrate with Binance live trading websocket APIfor production deployment of algorithms
- Integrate more technical indicators - information that can be used to guide trading decisions.
- Support more types of cryptocurrencies for a portfolio management solution
Log in or sign up for Devpost to join the conversation.