As a team, we all wanted to learn about what an API is, how it works and how to use one for meaningful purposes. After discussing, we found that we were interested in how to present data in a visual and engaging way. We decided to create a project that could be used to analyse current trends across whatever topic the user wanted to track.

We created a website which finds recent tweets containing a user-chosen input. It outputs two things: a word cloud containing the most common words and also a bar graph showing the polarities. This allows us to see how people on twitter feel about a certain topic.

After splitting into two groups, we worked on the front end and back end separately. The front end team began by trying to use Velo by Wix, but we had difficulty with the functionality as it was quite restrictive. We ended up creating a search bar using HTML and the anychart library to create the word cloud and bar chart using JavaScript.

The backend team started by playing around with the Twitter API to find out how it works. We began by creating a list of recent tweets, filtered by a user input and then progressed onto filtering the words in these tweets. It was difficult to make the list representative of the sentiment of the tweets. We got rid of filler words, punctuation and words that didn’t appear frequently. We created a dictionary containing each word and its frequency, for use in the word cloud. We also measured the polarity of each tweet for use in our bar chart.

One problem we faced was using pagination: twitter only allowed us to generate 100 tweets per query, but we needed a larger sample size. We initially tried to write our own loop that followed the next token to make requests for the next pages, but realised that it wasn’t as performant as using tweepy’s paginator class. We also struggled to integrate the front and backend due to a CORS error. No one on our team has had any experience in this area, so we had to research a solution and eventually found that setting a header on the web server fixed the issue.

We learnt so much during the day and really enjoyed the process. We learnt how to work as team and communicate with each other but also how to work with APIs, natural language processing, sentiment analysis and how to integrate the front-end and the back-end together. We also learnt how to code Word Clouds!

We’re really proud of the integration of the front and backend of our project. We’re also very happy with the way in which the data is presented in the word cloud and bar chart. If we’d had more time, we’d have added more analytics, for example location data and the frequency of the tweets. We’d also have liked to make our program a little faster.

Share this project: