My partner and I used to live in Russia where the online culture of hate, trolling and abuse is widespread, and I personally know victims who believe that something is wrong with them rather than with online trolls. How can we make sure that we don't deploy censorship while still keeping constructive online dialog? Detoxify addresses this problem in the user-friendly way. Identifying the problem is the first step towards solving it

What it does

It is a Chrome extension which detects, classifies and highlights toxic comments in English under Youtube videos using Deep Learning techniques.

How we built it

The core of our application is an LSTM-based custom model trained with PyTorch. We used the dataset from the Jigsaw Unintended Bias in Toxicity Classification. Then we deployed our model and built the API on PythonAnywhere using Flask. The user-facing side of our application is Chrome extension, which was build with Javascript (with bits of HTML and CSS) and Chrome API.

Challenges we ran into

It was our first time building a fully-fledged application from scratch, so we had a handful of technical challenges :) For example, before the hackathon we had no idea how to create Chrome extensions (first time working with Javascript as well) and how to establish communication between our model and our app. Also, a big challenge is speeding up the training process, because speed is critical in order to fully utilize the power of recent NLP advancements. We experimented a lot with pretrained BERT models provided by Hugging Face but in the end the LSTM-based model proved to be more fitting to our use-case due to its robustness and lightweight.

Accomplishments that we're proud of

We are proud to have been able to build an end-to-end application with intuitive design in quite limited amount of time. It addresses a major problem online which is very dear to us. We were really inspired by many interviews of activists on feminism, religious understanding, sexuality, and multi-culturalism. They are truly making a difference in this world. With this Chrome extension we hope to provide our tiny contribution.

What we learned

  • how to combine different parts (front-end, back-end, model training and updating) together
  • how to work with Flask and technologies from front-end stack
  • how to manage time properly to evenly distribute the load over specific period :)

What's next for Detoxify

We are planning to do the following:

  • improve the model performance
  • augment the dataset with platform-specific data (youtube comments, facebook comments etc.)
  • employ the state-of-the art techniques (such as BERT)
  • work with with languages other than English
  • extent the functionality (comment filtering or masking etc.)
  • detect passive aggression (that would be cool!)
Share this project: