After the release of Facebook reactions, we realized they could be used to get a more nuanced view of the reception of a particular news topic.

What it does

The user inputs in a trending search term and the input used to search Facebook for related posts. Our scraper then sorts the reactions of each of the top articles, then pushes it to the cloud. Our website then graphs this data in real-time every 30 seconds.

Challenges we ran into

Plotting a graph in real time was challenging. Many Javascript charting libraries were difficult to understand, especially at 3 in the morning. Scraping data from Facebook was also difficult since captcha tests were thrown multiple times. Parsing through entire HTML files for specific elements sucked since their selectors did not remain static.

What we learned

Scraping from websites is hard, especially when you need authentication from the website. We also learned how to use regex to parse through HTML files since we could not inject jQuery into the Facebook page. We had to learn Canvasjs for plotting data dynamically.

What next

Hopefully next time, we can connect the database to the webpage.

Share this project: