Visualise the news

An attempt to provide an intuitive way to visualise current news stories - in a word map where the viewer can interpret the relevance and sentiment of various entities being talked about in the news.

Comment

Inspiration

With so many devastating stories/situations going around us such as covid or Russia at the brink of war, it is important to grasp an understanding of what is gaining traction early - especially on the negative side.

What it does

Our platform scrapes the top headlines of popular news sites and uses the collected data to detect main entities. Each entity is given a certain score in its respective headline regarding sentiment and salience, which helps us calculate how heavily negative or positive this respective entity is in the news article that we scraped. When aggregating these scores, we are then able to compute a colour shade and size for the entity when displayed on our word map on the main page. The user is also then able to click on the entity (word) in order to gain access to all the articles we have scraped in order to come to our conclusion, introducing transparency.

How we built it

We decided to use Next.js as it is very easy and quick to set up, especially since we have used Node.js based frameworks in the past. The fact that Vercel allows us to deploy applications for free when using Next.js was an added motivation. We scraped the news data by querying the News API for the top headlines in the UK. Using the response from this query, we would use Google Cloud's Natural Language API to analyse the sentimental score for the headline tile and entity analysis for each entity afterwards. We would aggregate the results using a map structure in Node.js, where the 'entity' was the key of this data structure. We made sure to lemmatize the entity words in order to prevent duplication of certain enitities, which would lead to an inefficient word map.

Challenges we ran into

Google Cloud's authentication flow was very ticky to handle, especially since none of us had experience with their systems. It made it even harder for us to deploy the web app for wide use since it requires the use of a download JSON file that would be a security risk if it were to be uploaded along with repo.

Accomplishments that we're proud of

We are proud of our ability to manage multiple APIs within our backend with relative ease and learn a bit more about Google Cloud's services and specifically Natural Language Processing. It also introduced us to the world of data aggregation and visualisation which is an area we would like to explore further into the future.

What we learned

The main technology we have learnt is Google Cloud's services as well as front end development for data visualisation.

What's next for Visualise the news

We would like to figure a way around the authentication problem described earlier in order to deploy the web application for wide use. We would also like to start introducing flexibility to our product, where the user can query about specific countries or topics.

Built With

Share this project:

Updates