SUDO STOP_ABUSE

One stop solution for the reporting harassment on Facebook

Comment

Inspiration

Sudo Stop_Abuse - Stop abuse and harassment at all levels!!

It all started from the moment we heard about the cyber anti-harassment focus during the opening ceremony of hackgt 2016. We decided that this is an area which is of prime importance. Many people across the country and across the world, especially teenage kids are vulnerable to this kind of cyber bullying and harassment.

Facebook is an online social media platform which allows users to post, comment and message one another. Facebook allows users as young as 13 to create an account on their social media platform.

According to marketland statistics, almost 94% of teenagers use Facebook regularly. Public shaming and public harassment damages the life and careers of these teenagers and they often go into depression. One million children were harassed, threatened, or subjected to other forms of cyberbullying on the site in the past year.
Here are some statistics about harassment on Facebook. (courtesy www.pewinternet.org)

  • 27% of internet users have been called offensive names
  • 22% have had someone try to purposefully embarrass them
  • 8% have been physically threatened
  • 8% have been stalked
  • 7% have been harassed for a sustained period
  • 6% have been sexually harassed


Lots of instances of these go unreported because of the following reasons
  1. The victim is unaware of the fact that he/she is currently facing harassment
  2. The victim does not report it to his/her parents/friends because of the consequences


We wanted to build something to alleviate this and make sure that someone responsible is aware about the harassment.

What it does

Our project is a google chrome extension. This chrome extension called SUDO STOP_ABUSE will make you, as 'parents' aware whenever your child is facing online harassment.
Once you install this chrome extension and enter the parent's desired contact information(email-id), it will start monitoring your child's facebook post, messages and comments on the posts that he is tagged in.

How we built it

The Facebook graph API was used to read posts and comments pertaining to a user's account in a non-intrusive way (such that children/teens wouldn't even know its running) and is classified as either abusive or not using the Bark Partner API (https://partner.bark.us/).

The reason for choosing this API was the level of detail about a text that is returned - Abusive or not, in what context(profanity, abuse etc.) and level of abuse(likely, very_likely etc.).

We used a PHP backend server that serves the page behind the extension which refreshes at frequent pre-set intervals monitoring new content from the facebook feed.
Tech used in building the extension: JSON, HTML, JS, PHP

We also trained our own model for text(harassment/not) classification using scikit-learn (Vectorizing - TfIdf - Support Vector Machine) pipeline with Grid Search for optimization and trained on #HackHarassment data(https://github.com/HackHarassment/TwitterClassifier/tree/master/data).
Our model significantly out performs the benchmark given in https://arxiv.org/ftp/arxiv/papers/1609/1609.02809.pdf. (Precision - 0.90, Recall - 0.88). (#Note: Possible overfitting due to small amount of bad_corpus available.) We could not integrate this model in the application due to time constraints.

Challenges we ran into

These are the challenges we ran into while developing the extension.

  • Facebook only allows apps/extensions which have not yet been reviewed to request for only posts and comments related to the user. It does not allow us to access the inbox of the user without us submitting our extension to Facebook. After 3-7 business days, Facebook will grant permission to your app to access inbox. So currently, we are limited to finding harassing posts
  • We used the bark api to find out whether a sentence is abusive in nature. But we ran into "CORS issue. No ‘Access-Control-Allow-Origin’" which made it impossible for us to access their api from our server. This was a web security issue and we circumvented this by sending a curl request through php.

Accomplishments that we're proud of

In most cases, even after reporting abuse, Facebook would be able to handle the case within 72 hours and remove the post. In rare and serious cases, Facebook will inform law enforcement agencies.

However, As Facebook explains:

Because of the diversity of our community, it's possible that something could be disagreeable or disturbing to you without meeting the criteria for being removed or blocked. For this reason, we also offer personal controls over what you see, such as the ability to hide or quietly cut ties with people, Pages, or applications that offend you.

But, using the extension, a friend/peer/family knows about the situation/issue that the victim is going through and is able to provide support. Moreover, in cases of children, it is vital that their parents know what their kids see and share, if not receive, in social media

The extension detects any potential harassment and reports it to the designed point of contact.

The most important accomplishment is the satisfaction of potentially supporting what could be millions of users on facebook and providing them moral support which can not be quantitatively valued.

What's next for SUDO STOP_ABUSE

The next steps for SUDO STOP_ABUSE are the following

  1. Once Facebook finishes review and approves, we will also be able to monitor the messages in the inbox with 0 to minimal change in our extension.
  2. We are also planning to port this extension to other browsers as well such as Microsoft Edge, Safari and Firefox.
  3. A mobile app which performs the equivalent of sudo stop_abuse.
Share this project:

Updates