Hate speech, aggressive language, and cyberbullying on social platforms can make the experience of being digitally immersed very difficult. While the argument of having the freedom of speech continually persists, the lines between true freedom and offensiveness become blurred. Freedom of speech can easily be warped into offensive, hateful, and unconstructive words online, particularly towards people who belong to marginalized communities. Formally, hate speech can be defined as abusive or threatening speech or writing that expresses prejudice against a particular group, especially on the basis of race, religion, or sexual orientation (as defined by Oxford Languages).
As citizens, how can we keep track of hate speech online that's affecting our fellow peers and neighbors? I think everyone has a responsibility to keep each other safe and speak up against hateful actions. While I believe there are myriad solutions to helping each other out, I wanted to try a solution using machine learning models. Machine learning classifiers, alongside a vast amount of data gathered through API calls, can offer valid solutions to organizations and companies attempting to monitoring content on their platforms.
What it does
The main goal of this project is to build a model that discern hate speech on Twitter, a platform that rapidly lets your thoughts out with a simple click. I have tried to follow a typical machine learning cycle in order to generate my model. The offline logistic regression classifier that I have built classifies language into two categories, offensive and non-offensive, with a current 98% accuracy.
How I built it
In this project, various classifiers such as logistic regression, decision trees, and convolutional neural networks were applied to a pre-processing dataset in an attempt to classify language used in tweets. The tweets themselves, in total 40,000 of them, were labeled as negative (offensive) or positive (non-offensive).
In addition, it's important to make sure that the model receives data that is clean and relevant. For this reason, it is important to make the following modifications:
- Lowercase all words in tweets (eliminate any bias that could stem from words being uppercase or lowercase)
- Removing duplicate tweets
- Removing retweets
- Removing Twitter handles
- Removing mentions
- Categorizing the parts of speech (lemmatization)
- Removing excess whitespace
- Removing stop words and words that are two characters or less
Making all these modifications causes extra columns to be added to the data frame as we process through. In the end, we drop the irrelevant columns.
Tweets can be tokenized and we can perform further operations on them, such as changing "n't" to "not". More detail can be found in the
Exploration.ipynb file in my repository linked below. We can look at the most common negative and positive words in a given dataset as well.
The only predictor used for the modeling is the pre-processed and lemmatized version of the text. We use a Term Frequency Inverse Document Frequency (TF-IDF) vectorizer to accomplish this. This is a common algorithm used to transform the text into a meaningful representation of numbers used to fit the machine algorithm for prediction. The TF-IDF object is pickled so that it can be used in the analysis of newer tweets later on. The threshold of the maximum and minimum document frequency has been set at 90 and 20 percent, respectively. The deep learning model also requires different kinds of preprocessing and that will be applied right before the CNN modeling.
The TF-IDF matrix was used across all of the models except for the CNN and Naive Bayes. The CNN performance had a very low accuracy (just above 50%) on the validation set. The neural network definitely needs improvement in order for deep learning to be a robust model for this application.
The high performance of logistic regression is due to the fact that there is no neutral class of tweets present like there was in the unmodified datasets. If we were to introduce a third class of neutral tweets in addition to the positive and negative tweets, the performance of the logistic regression has a definite chance of decreasing inaccuracy.
Challenges I ran into
The model created is currently offline and has not been deployed yet. Currently, I have applied for Twitter developer access in order to secure API keys and create a live feed upon deploying the logistic regression algorithm. It usually takes 2-3 days to get approval from Twitter. However, once this is done, I do have more plans to work on this application and deploy my model!
Accomplishments that I'm proud of
Despite not being able to deploy my model because of some factors that aren't in my control, I am proud of myself for being able to find datasets, clean and preprocess them, do some exploratory data analysis, and be able to run various algorithms on my data, including logistic regression, Naive Bayes, a CNN, and decision trees in a short period of time during this hackathon. Data preprocessing and understanding data takes up significantly more time than actually creating or deploying a model, so I was particularly happy that I was able to accomplish that goal!
What I learned
I learned a lot about how language impacts our daily interactions. By being able to apply the NLTK functions and learn more about natural language processing, I was able to understand just how powerful NLP as a field is and I want to continue to learn more about it. I learned more about the process of deploying a machine learning model online and how I can integrate new data into the model by using API calls.
What's next for Twitter Hate Speech Detector
Once the Twitter API is connected, fresh tweets can be collected periodically (perhaps every few hours) on a local MySQL database which solely serves the purpose to house incoming sets of tweets. The Python library Tweepy can be used to create a connection to the Twitter API. Within my current codeline, I've included a file known as
custom_tweepy_listener.py that would serve this purpose. It gathers the necessary information about an incoming tweet (actual tweet content and time). All text would need to trim of emojis in order to be stored in the database. It's possible to also filter by topic. Specifically, we can look at topics potentially related to Islamophobia by looking at keywords such as "Islam" and "Muslim." We gather this specific information by looking at the hashtags on tweets.
With this pipeline in place, we can pull sets of data from MySQL to a new Jupyter notebook where it can undergo similar preprocessing to the dataset we had cleaned before. Ideally, it would be great to derive insights into the top most offensive words being used on the platform and the ratio of negative to positive/neutral content circulating on the website.
Currently, this application was is displayed using Flask and deployed on Github Pages. It is possible to deploy the machine learning model via Flask to allow a live feed of data analysis being done as new tweets come into the database and go through the preprocessing and algorithm via Heroku.
I hope that I will get developer access to those API keys, and I will be actually able to deploy this model online in the near future.