Hol' Up was inspired by the current political climate and the often toxic posts released on social media. We wanted to create a tool that acts as "an angel on your shoulder" and advises users to not send potentially offensive messages online.
What it does
Hol’ Up is a chrome extension that aims to mitigate toxic behavior and hate speech on social media platforms. The detection of hatred can be a grey area as one cannot draw the line as exact as positive vs. negative, but rather the differences between non-hatred and hatred are more nuanced. Hatred detection is an ongoing area of research, where even the state-of-the-art methods suffer from interpretability problems and out-of-sample inaccuracy. We tackle this online issue and employ keyword detection using offensive words from the HateBase API and an AI-backend to reinforce our hatred detections, which appear in the form of a popup window.
How we built it
For the frontend, we use HTML/CSS/JS and a novel injection of a customized event handler to hijack send button capabilities and instead displaying a popup html page acting as a cool-down for the user. This acts as a way to give them time to think about their decision and deter them from sending potentially harmful messages.
For the backend, we used a lightweight neural network with an incorporated Tensorflow Hub Word2Vec embedding, which was trained on the publicly available datasets Jigsaw/Conversation AI and Hatred on Twitter during the MeToo Movement. We deployed our model for fast predictions to GCP as a flask application.
Challenges we ran into
A major challenge for the front-end was being able to intercept button actions after the user clicks the send, whether it be on Twitter or Reddit. For the backend, we were trying to develop an accurate hatred detection model. Given that the detection of hatred is still an ongoing research area in need of improvement, we had to tackle the nuances of detecting subtle hatred hidden in messages. Because the hackathon is only 36 hours and that predictions should ideally be done in real-time, we couldn't train state-of-the-art models such BERT in a practical amount of time on large amounts of data before the deadline.
Accomplishments that we're proud of
We're proud of the event hijacking that was achieved on the front-end. On the backend, we are happy about our fairly decent hatred detection model and the usage of HateBase's API to retrieve double meanings.
What we learned
What's next for Hol' Up
Our chrome extension is currently giving users a cool-down before sending hateful messages online. This aims to prevent hateful messages from being sent in the first place. However, if they've already been sent, we can consider adding a feature to warn users of biased or hateful content. This warning would be in the form of a popup as well. This can make our application more robust and allow the user experience online to be safer. We could also expand our project by giving the users more flexibility in terms of understanding what's contained in, for instance, reddit posts before reading them with visualization tools such as Chart.js.