When we open news app or news website, the news that we see are mostly ingested in our flavour from our previous reading history. For example if a user supports the conservative party, he/she will be more likely to read news from conservative outlets. As the result, the reader will get "stuck" with a relative narrow range of perspectives and will not be likely to get expose to the point of views from the other side.

Therefore this inspired us to design and build this plugin that gives reader 3 articles recommendation based on the current article that the reader is reading, but which may share a different point of view.

What it does

when the news reader click onto Oppostasy on a news article site, a side bar will show up on the right side of the window that shows a list of 3 news article with opposing views with title, picture from the news with a clickable link that can redirect reader to the suggested news.

How I built it

The plugin has two main areas the front-end that is implement with javascript flask, and the backend news pipeline to ingest recommending news implemented with python. The news pipeline consists of two main part: 1) parse the current news article that reader is reading with combinations of natural language processing algorithms including TF-IDF, RAKE and existing nlp libraries. 2) ingest list of news from opposing news media outlets using the information we collected from the first step. Then the result is output to the front end and presents in front of the readers.

Challenges I ran into

One of the biggest challenges that we faced is in the design of the nlp algorithm, which extracts keywords from the current article. We initially used TF-IDF formula that calculates the importance of a word in a document within a collection of documents. The issue is that we are only analyzing the article that the reader is currently reading instead of a collections. As the result, we tweaked the algorithm by changing it from article-base to sentence-base, and incorporates the idea of RAKE algorithm for single document keywords extraction.

Accomplishments that I'm proud of

We are proud that we have completed the plugin considering that we don't have previous experience working on Google Chrome plugin and have no prior knowledge in nlp.

What I learned

We learned how to crawl the web with a python html scraper. We have also learned how to search a website for keywords so that a computer can understand what the website is about. Our understanding of chrome extensions has also been greatly enhanced.

What's next for Oppostasy

There are three areas for Oppostasy that we can work on next. 1) support more news site so we can get more varieties in the recommandation output 2) deploy the server 3) improve the accuracy in nlp algorithm

Share this project: