Inspiration

Falling for facts we learn from that tweet or social media post by our favorite twitter celebrity. Our world has changed significantly from past when the only source of our news was News Papers. Pew Research reports that as of August 2017, two-thirds (67%) of Americans report that they get at least some of their news on social media – with two-in-ten doing so often. enter image description here

And as we see more than 74% of it comes from twitter enter image description here

But there is no accountability to what gets posted in twitter. We cannot be sure of what we read on the social media sites, and often that sparks heated debates across us and our friends on a topic that should have been clear from the get go.

The Problem

  • Our inability to check a tweet for facts.
  • Our inherent bias wants us to believe anything that favors our opinions and views, even if thats factually not correct. Hence preventing us from fact checking it.
  • Social media sites algorith for engaging us traps us into filter bubbles without giving us an easier way to differentiate facts from fiction ## What it does

This is a browser web extension, runs both from Firefox and Chrome, and it checks any tweet apperaing in yout twitter timeline for facts and shows you related articles to help you get a broader idea about the tweet's content and facts

Internals

"Related Facts" is based on two components

  • One is the browser extension which adds a "RF" button to every tweet. It merges with the existing twitter UI seamlessly and doesn't stand out or distract. However when a user clicks on the button it sends the tweet to the server for being processed

  • The other is a Machine Learning based Web API that processes the tweets and gives us with a score of how relaible that is along with related news articles.

How I built it

The whole solution consists of a browser extension (to scrape the tweet and add the button), a Flask server that connects with the Twitter API, a news search API & a NLP pipeline built on top of the Google Cloud API. I calculate the final ‘scores’ for the news articles and then show the related ones.

It is a Machine Learning based web API that allows users to fact-check by getting related news articles for any tweet on twitter. The service helps in getting facts right by suggesting related news articles and Wikipedia entries that aim to give a better understanding of the tweet.

  • The NLP pipeline performs entity and topic extraction from tweets to search for news articles and discover Wikipedia articles. I utilized Google CLOUD NATURAL LANGUAGE api to analyze the text and extract entity and classify content

  • This information is then processed to detect the *sentiment * to get features for the tweet as well as the news articles which we use to select news articles from news pool

  • For news I used the newsapi to get breaking news headlines. The top three articles to show are selected based on the previous step

  • It is then pushed back to the twitter page to show to the user

Challenges I ran into

In every aspect. Google Cloud Natural Language API is not free. Had to learn ho to use that. Failed to actually implement it in my own server, so for now the server is running in my local machine. For that exact same reason could not publish the extension to Firefox Add-On or Chrome Extension store

Accomplishments that I'm proud of

It mosttly works. And relatively fast. Also very happy that it works in both Firefox and Chrome

What I learned

Don't try a new hackathon project in a new windwos laptop when you know you are going to use python. Also Google Cloud API's are way cool.

What's next for Related Facts

  • Fix the ssl issue in server and implement it in my web server
  • Publish the browser extension
  • Add a secondary ranking mecahnis using Azure web services for entity recognition. I am not really happy with what I am getting with Google
    • *Extend this to include Facebook *
Share this project:
×

Updates