Escape The Rabbit Hole

A web application that takes an excerpt of any article/post/blog and displays the user with similar articles from trustworthy news sources on both sides of the political spectrum.

Comment

Inspiration

In a world so full of information, we are bound to be closer to the truth and reality, right? Why do we seem to be more polarized than ever, whether politically, socially, or religiously? How is it possible that we can’t even seem to agree on anything, solve any problems like climate change or discrimination? The problem stems from the very abundance of information we are consuming. With the more news and articles we read and intake, we begin focusing on opinions matching our own perspectives.

We call this getting stuck in the rabbit hole. A hole where your own views are echoed back to you repeatedly until any other perspective seems crazy. And these rabbit holes are destroying the fabric of society.

Now imagine if someone could pull you out of that rabbit hole and show you a new way to consume news. One that’s balanced, with perspectives from all over the political spectrum, with trustworthy and reliable information.

Well, look no further. Escape the Rabbit Hole is a simple way to improve your news diet.

What it does

Escape The Rabbit Hole’s primary goal is to help users diversify their opinions, open themselves up to various perspectives, begin to understand other people. The web application takes the title or excerpt of an article and uses a keyword extraction machine learning algorithm to determine the most relevant keywords. From there, the keywords are inputted into the Google Custom Search JSON API to find related articles from trusted, hand-picked news sources from across the political spectrum.

How we built it

  • We began by using the Sklearn python module along with the sentence_transformers packages to build a machine learning algorithm that extracts the keywords/phrases from a given text
  • To determine the similarity between text and keywords for the best outcome, we used cosine-similarity to calculate the differences between vectors (converted from the texts).
  • To search Google for related content and details, we configured the Google Custom Search JSON API to loop through pre-defined news sources such as CNN, NBC News, New York Times, etc. For diversity, we decided to output 10 sources with 5 on each side of the political spectrum.
  • For a connection between our Keyword Extraction Algorithm python file and our main Javascript file, we used a Flask server that acted as a REST API.
  • As data returned from the Google Custom Search JSON API enters our main Javascript file, we used the JSON package to parse for important information (title, summary, link).
  • We used HTML, CSS, and Javascript to design a minimalism-inspired UI/UX design.
  • For efficient and effective UI/UX design we used Bootstrap.

Challenges we ran into

  • Initially, we were looking into creating a Chrome extension for the project, however security protocols were too tight to allow any APIs to pass. As a result, we decided to opt for a website with the same functionalities.
  • Having no prior experience with machine learning, we learned the BERT keyword extraction method and wrote the algorithm.
  • Likewise, it was also our first time using Flask. As such, we had to learn it from scratch. Specifically, we had to learn how to make POST requests and responses to the server for connection between the main Javascript file and the Keyword Extraction Algorithm script.

Accomplishments that we’re proud of

  • We are without a doubt proud to have created this project with elements we’ve never learned.
  • We managed to learn Flask within a day and implement it into our project.
  • We learned to integrate various ideas into Escape The Rabbit Hole as stated above.
  • Finishing the project in less than 48 hours!!

What I learned

  • Basics of machine learning algorithms and keyword extraction
  • The usage of Google Custom Search JSON API
  • How to communicate between server-side and client-side languages.
  • How to read, parse, and manipulate JSON data.
  • The use of Flask, POST response and request calls

What’s next for Escape The Rabbit Hole

The journey with Escape The Rabbit Hole does not stop here. We have yet to begin our development in Browser extensions (which we are highly excited about). With an extension, we will no longer need users to paste their text into an input box; rather, a web scraper will be implemented to retrieve the data automatically. In addition, we are looking to create a database to store recent searches and results that the user can access. These additions will significantly increase the UX of Escape The Rabbit Hole.

Built With

Share this project:

Updates