-
-
-
-
Multiple summarizations
-
Political bias scale for ML training dataset
-
-
Visualization of NLP ranking graph
-
-
-
-
-
Inspiration
The outcome of 2016's election made it clear to our team that the media and the role it plays in informing voters is crucial in politics. This, in addition to the increasing prevalence of partisan news coverage as well as the confusion surrounding verifiable news resources lead us to create Spectrum.
What it does
Spectrum is a chrome extension which provides greater information about linked news articles. Upon hovering over a link to an article, it first categorizes the linked article by political leaning, then summarizes the article in three sentences, and finally provides articles from different political leanings covering the same topic.
Political Bias
The political leaning of an article is determined through two methods, the first is utilizing a public database allsides, which indexes certain news articles and sources with a political bias rating from -2 (liberal) to 2 (conservative) with 0 being neutral. If an article cannot be found on allsides, we developed a backup machine learning algorithm to determine political bias. The algorithm is trained against the corpuses of news sites with political leanings defined by the organization Pew Research.
Summarization
We developed a custom NLP algorithm from scratch to summarize article text. Article text is first scraped utilizing python's newspaper library. The algorithm then separates each sentence into tokens, then ranks sentence similarity based off of a cosine similarity matrix generated utilizing TF (term frequency) X IDF (inverse document frequency). An undirected graph is then constructed with each sentence acting as a node and each weighted value acting as an edge between each node. We then apply Pagerank, the same algorithm Google uses to determine page importance in a graph, to this unweighted graph in order to rank sentences. The top three sentences are then selected and rearranged in chronological order, giving us a summary of an article.
Recommended Articles
Spectrum searches for articles with similar keywords (particularly in headlines). These similar articles are then also indexed by political leaning, and articles with opposing political stances from the article currently displayed are recommended to the reader in order to expose the reader to differing viewpoints.
What's next for Spectrum
More accurate political bias rankings and NLP summarization!
Log in or sign up for Devpost to join the conversation.