CyberCensor

Hate speech detection on social media

Comment

Inspiration

In a world increasingly connected through social media platforms, hate speech through social media is becoming one of the things to worry about. As we move ahead, Automation is quite necessary because of the increase in the number of users. The idea was to build something useful and unique for the community. To understand the use case let's walk through an example. To keep it intuitive let's consider one of the well-known YouTubers "MrBeast". MrBeast has around 166m subscribers and his videos have comments more than 100k. Now as a YouTuber, there are instances where the audience does not like the video and sometimes turns toxic and gives hate, and in response to that Youtubers to avoid hate switch their comment section off. But for a YouTuber like MrBeast who has 100k+ comments, it's practically impossible to go through all the comments and make the decision to switch off the comment section or not. This web application can make life easy. Bring the video URL here and upload it, the automation process will run and it will scan through all of the comments and based on the ML model identify hate speech and gives you stats on bases of which you can make the decision. This is just one use case. The idea was to automate the hate speech identification process on social media.

What it does

The project aims to automate the process of hate speech detection on social media platforms, specifically targeting tweets and YouTube comments. By providing a link to a tweet or a YouTube video, the system retrieves the content and performs hate speech detection using a machine learning model.

For tweets, the system analyzes the text of the tweet to identify any instances of hate speech or offensive language. It provides an assessment of whether the tweet contains hate speech or not.

Similarly, for YouTube videos, the system scans through the comments section, analyzing the content of each comment. It applies the machine learning model to classify comments as either clean or flagged as hate speech. It also provides statistics indicating the number of clean comments versus the number flagged as hate speech.

How we built it

Firstly, I developed the front end of the website using React.js, a JavaScript library that allowed me to build an intuitive and interactive user interface. I integrated Firebase into the application, utilizing its authentication feature to enable users to log in securely using their Gmail accounts. I also leveraged Firebase's Realtime Database to store and manage the data generated by the project. Moving on to the backend, I implemented two APIs using Python and Flask.

The first API was responsible for fetching the data from the provided social media links, retrieving the content of tweets or YouTube comments. The second API incorporated a machine learning model trained to detect hate speech. It took the retrieved content as input and applied classification techniques to determine whether the content contained hate speech or not. The result of the classification was then returned to the front end, where it was displayed to the user, indicating whether the content was clean or flagged as hate speech. By combining these technologies and components, I was able to create an automated hate speech detection system for social media.

Challenges we ran into

There are numerous challenges we ran into

  1. One of the main challenges was which ML model to select. As this is a classification-based model there were multiple options to select the ML model. After a careful evaluation, we used brute force to calculate output accuracy for each algorithm and chose the winner which in our case was Navie based Algorithm
  2. One of the biggest tasks was to make local APIs with Flask and integrate it with the web app. I accomplished that by assigning different ports to different APIs and making a call to them from the web app.
  3. As the software was hosted locally because of the system rules it became quite challenging to make calls to API because of the concept called CORS. CORS stands for Cross-Origin Resource Sharing, and it is a security mechanism implemented by web browsers to restrict resource access between different origins (domains, ports, or protocols). When an API request is made from a web application running on one domain (e.g., port 3000) to an API server running on another domain (e.g., port 8000), the browser enforces the same-origin policy by default. This policy prevents the web application from accessing resources on a different origin for security reasons. This issue was resolved by enabling CORS on the API server.
  4. Finding the correct blend of the data set was also one of the challenging parts as different data sets affect the accuracy the model produces. After examining multiple data sets the data set which was giving the best accuracy and precision was selected.

Accomplishments that I am proud of

First and foremost I am proud of achieving my use-case of the web app. Developing a hate speech detection system that contributes to creating a safer and more inclusive online environment. By flagging and addressing instances of hate speech, you are actively working towards combating online harassment and fostering positive interactions on social media platforms. The project achieved an accuracy of 83% while classifying the text. Moving ahead the backend part is connected to the Firebase cloud which maintains the scalability and efficiency of the application.

What I learned

Throughout the course of this hate speech detection project, I had the opportunity to gain valuable technical knowledge and skills. Here are some of the key learnings from the project:

  1. Machine Learning: Developing a hate speech classification model allowed me to dive deep into machine learning techniques. I learned about data preprocessing, feature engineering, and model selection, enabling me to train and deploy an effective model for identifying hate speech.
  2. Natural Language Processing (NLP): Implementing NLP techniques for text analysis provided me with valuable insights into the world of NLP. I gained hands-on experience in tasks such as tokenization, stemming, and sentiment analysis, which proved instrumental in understanding and detecting hate speech.
  3. Web Development: Building the front end using React.js and integrating Firebase for authentication and Realtime Database enhanced my skills in web development. I gained knowledge in handling API requests, managing user authentication securely, and leveraging cloud-based solutions for data storage.
  4. Data Handling and Preprocessing: Working with large volumes of social media data required me to develop proficiency in data handling and preprocessing. I learned techniques to clean and preprocess noisy data, ensuring the accuracy and reliability of hate speech detection.
  5. Flasks: Building Apis through flask was a skill I added to my portfolio in this project.

What's next for CyberCensor

The is a list of goals to be followed-

  1. Improving Accuracy - Implementing steps like Negation Handling and improving the data set will help in increasing the accuracy.
  2. Feature addition - Add a feature where if i list of links is provided it detects and responds to all
  3. Buy a domain and go live.

Built With

Share this project:

Updates