Falling for facts we learn from that tweet or social media post by our favorite twitter celebrity. Our world has changed significantly from past when the only source of our news was News Papers. Pew Research reports that as of August 2017, two-thirds (67%) of Americans report that they get at least some of their news on social media – with two-in-ten doing so often.
And as we see more than 74% of it comes from twitter
But there is no accountability to what gets posted in twitter. We cannot be sure of what we read on the social media sites, and often that sparks heated debates across us and our friends on a topic that should have been clear from the get go.
- Our inability to check a tweet for facts.
- Our inherent bias wants us to believe anything that favors our opinions and views, even if thats factually not correct. Hence preventing us from fact checking it.
- Social media sites algorith for engaging us traps us into filter bubbles without giving us an easier way to differentiate facts from fiction ## What it does
This is a browser web extension, runs both from Firefox and Chrome, and it checks any tweet apperaing in yout twitter timeline for facts and shows you related articles to help you get a broader idea about the tweet's content and facts
"Related Facts" is based on two components
One is the browser extension which adds a "RF" button to every tweet. It merges with the existing twitter UI seamlessly and doesn't stand out or distract. However when a user clicks on the button it sends the tweet to the server for being processed
The other is a Machine Learning based Web API that processes the tweets and gives us with a score of how relaible that is along with related news articles.
How I built it
The whole solution consists of a browser extension (to scrape the tweet and add the button), a Flask server that connects with the Twitter API, a news search API & a NLP pipeline built on top of the Google Cloud API. I calculate the final ‘scores’ for the news articles and then show the related ones.
It is a Machine Learning based web API that allows users to fact-check by getting related news articles for any tweet on twitter. The service helps in getting facts right by suggesting related news articles and Wikipedia entries that aim to give a better understanding of the tweet.
The NLP pipeline performs entity and topic extraction from tweets to search for news articles and discover Wikipedia articles. I utilized Google CLOUD NATURAL LANGUAGE api to analyze the text and extract entity and classify content
This information is then processed to detect the *sentiment * to get features for the tweet as well as the news articles which we use to select news articles from news pool
For news I used the newsapi to get breaking news headlines. The top three articles to show are selected based on the previous step
It is then pushed back to the twitter page to show to the user
Challenges I ran into
In every aspect. Google Cloud Natural Language API is not free. Had to learn ho to use that. Failed to actually implement it in my own server, so for now the server is running in my local machine. For that exact same reason could not publish the extension to Firefox Add-On or Chrome Extension store
Accomplishments that I'm proud of
It mosttly works. And relatively fast. Also very happy that it works in both Firefox and Chrome
What I learned
Don't try a new hackathon project in a new windwos laptop when you know you are going to use python. Also Google Cloud API's are way cool.
What's next for Related Facts
- Fix the ssl issue in server and implement it in my web server
- Publish the browser extension
- Add a secondary ranking mecahnis using Azure web services for entity recognition. I am not really happy with what I am getting with Google
- *Extend this to include Facebook *