Uniter Bot - RL crypto trading bot +45% profit

Uniting multiple news source and twitter data with BERT sentiment analysis to feed into a PPO reinforcement learning algorithm to trade crypto (ETH-USDT). Achieving a 45%+ return.

Comment 1

Inspiration

We set out on a mission to unify multiple datastreams in an effort to gain a competitive edge in the world of crypto trading. Our advantage over other programs is Reinforcement Learning (RL), an area of Machine Learning. RL agents have the ability to learn continually and dynamically mimicking what a real-life trader does. An RL agent has access to past price data, trading indicators and news sentiment analysis just like a trader, they then can make decisions in this environment: buy, hold, sell and learn from their mistakes creating the best possible trading strategy. This was our mission, and not to spoil the ending but we succeeded.

What it does

Our trading agent is a new take on the traditional trading bots leveraging the power of NLP, Reinforcement learning and multiple streams of data. See the image for an overview of how all the parts of the Uniter Agent fits together.

It takes in price data, news sentiment data as well as trading technical indicators and it outputs an action: buy, hold or sell for a specific crypto currency. We then run this through a testing period to analyse the performance of the agent.

How we built it

Binance API for all crypto trading data Technical trading indicators used: macd, boll_ub, boll_lb, rsi_30, dx_30, close_30_sma, close_60_sma

We leveraged state of the art pretrained language models (BERT) to identify sentiment from tweets and articles giving an insight into attitudes towards a particular coin throughout the year. The crypto market is known to be heavily speculative which made this the best market for us to deploy this technology. These sentiments were then aggregated over a 4 hour period.

Challenges we ran into

Trained and tested 3 models: Soft actor critic, DDPG and Proximal policy optimisation (PPO) With a range of hyper parameters: trading interval (5min, 1hour,4hour,8hour,1day) Training took between 30 and 45 minutes per test which meant we had to focus on only experiments we needed to run to learn more about how the agent and environment works.

PPO was the best algorithm for this and it is the one in the final agent.

The Twitter API was extremely hard to work with and limited us on the number of tweets that we could retrieve per hour and the search criterias that we could apply.There were some open source libraries but they were outdated and no longer worked, so it was a challenge to find one that fits our needs. However, after a lot of searching, we came across one library that had been built to scrape on a much larger scale and this allowed us to collect enough data to facilitate learning.

Another issue was that a lot of tweets were noisy. They would contain coin keywords but the main content of the tweet would be completely unrelated. To tackle this, we added extra cryptocurrency keywords to our search parameters to ensure that the tweets were on topic.

Scraping data from Google news also proved to be a challenge. Google news does not have an API meaning we had to find an open source library that directly scraped search results. Then using search filters ‘before’ and ‘after’ in google, news articles from precise dates could be scraped. We decided to only use the news headline and description of the article in analysis to reduce noise in results, increasing confidence in sentiment.

Accomplishments that we're proud of

Scraped over a year's worth of data from Twitter and Google News feeds regarding cryptocurrencies. Built and backtested a RL powered crypto trading agent in under 24 hours. Achieving 45% Profit!! HODL (Buy / Hold strategy) - benchmark

Statistic   Value
Annual return   0.070374
Cumulative returns  0.409945
Annual volatility   0.296061
Sharpe ratio    0.377606
Calmar ratio    0.178156
Stability   0.747045
Max drawdown    -0.395016
Omega ratio 1.068705
Sortino ratio   0.554558
Tail ratio  1.086825
Daily value at risk -0.036857

Our Agent - Uniter Bot

 Annual return  0.077438
Cumulative returns  0.455854
Annual volatility   0.285704
Sharpe ratio    0.403711
Calmar ratio    0.227455
Stability   0.762475
Max drawdown    -0.340454
Omega ratio 1.074838
Sortino ratio   0.598638
Tail ratio  1.107681
Daily value at risk -0.035538

What we learned

How to scrape Twitter given the recent removal of the Twitter API for non academic sources to get many thousands of tweets. How to scrape Google news page results to get thousands of articles Implementation of RL using sentiment data and trading metrics on a highly volatile crypto market with really good return results! How to preprocess textual data in order to extract the core content How to leverage large pretrained transformer models in order to understand sentiment How to tune hyperparameters for the stock/crypto trading environments - completely different the openai gym atari environments

What's next for Uniter Bot - RL crypto trading bot

  • Integrate with the Twitter API directly to access the most recent real-time tweets for use when making decisions.
  • Collect more news and Twitter data over a longer time period to give the agent more to learn
  • Integrate with Binance live trading websocket APIfor production deployment of algorithms
  • Integrate more technical indicators - information that can be used to guide trading decisions.
  • Support more types of cryptocurrencies for a portfolio management solution

Built With

Share this project:

Updates