This year alone, our team has spent hours scrolling through news applications, reading a stream of updates on the constant barrage of events in the world. We've seen firsthand how hard it can be to get a balanced view on a topic and how easy it can be to forget to even try. During these times, it's especially important that we have an easy and quick way to get information on a topic - with arguments coming from multiple perspectives.
To this end, we wanted to automate the process of searching for truth in the news - an application that automatically finds sources with varying biases, summarizing each position to allow the user to learn something about a topic without effort.
What it does
Our web application, SplitNews, allows the user to search for a specific topic, (for example, affirmative action), and quickly and easily see a summary of the topic, from both left and right leaning sources. We automate the process of looking for sources with varied biases, presenting the user with a list of articles from reputable sources from both sides of the aisle, as well as a short and a long summary for each of the articles. To spread awareness of implicit biases in news sources, each news source is color coded blue for left leaning and red for right leaning.
How I built it
We were able to construct our website using Flask for our backend and React in combination with HTML and CSS to construct our frontend. To distinguish biased news sources, we used a combination of website heuristics, and a custom trained transformer neural network using Tensorflow and Hugging-Face's Transformer on over 5000 news sources, achieving 75% accuracy. We used word and sentence tokenization in NLP to summarize the articles and we used the Bing News Search API to obtain links and relevant information to the articles.
Challenges I ran into
Getting a large enough dataset to train our neural network proved to be difficult - we ended up using Selenium and BeautifulSoup to scrape news articles from various sources, on a variety of topics, eventually ending up with about 5000 articles.
Accomplishments that I'm proud of
It was so much harder to coordinate with each other remotely - we're proud that we were able to work with each other through Zoom and of course we're especially proud of our Trello board.
What I learned
This project involved many first experiences for our team, from our first time working with React to our first time using the Hugging-Face transformer library.
What's next for SplitNews
For phase two, we're hoping to make a SplitNews Chrome Extension that automatically helps you any time you're reading the news, whether it's something you Googled or a link you received from someone else.