Social Media Analysis

We aim to provide a platform that brings together the feeds from several social media platforms with visualizations of sentiments of the post in a nice and friendly web app.

We learned to wrangle data, apply deep learning model to get the sentiment analysis and make the web application with fast API and react. We also learned to make an intuitive UI and visualize data in pretty pie charts and barplots. And most importantly, we learned to work in a team.

Documentation

Working with data

An easy to use app to visualize what people are talking on socialn media during an Oympics.

Features to be added:

see visualizations according to the locations of audiences
feature to let use specify the type of sentiment and number of tweets to read the tweets

For the sake of demonstration of the app, instead of getting live posts from the social media APIs, we used data of tokyo olympics 2020 that is found here

The EDA can be found in the tokyo_olympics0.ipynb file. The sentiments were predicted using hugging face pretrained model.

from transformers import pipeline

sent_pipeline = pipeline("sentiment-analysis", 
    model='finiteautomata/bertweet-base-sentiment-analysis'
    )

This model gives {sentiment, sentiment_score} for the input text. EG.

('NEG', 0.7984506487846375)

Precomputed sentiment and score columns are added to dataset for fast accessing and scalability. Now the data and the sentiments are ready to be consumed by the web app.

Web app

Backend

We have used Fast API for making our web app faster. We have three API endpoints:

/overall: It takes get request and returns title, total number of posts and a list with number of posts with negative, neutral and positive sentiments respectively.
/sentiments: It takes keyword and filters all the rows with the given keyword . It also returns title, total number of posts and a list with number of posts with negative, neutral and positive sentiments respectively.
/tweets: It takes keyword, number of posts and sentiment of the posts and returns the filtered posts.

Frontend

We have written frontend on react js and used chart js for visualization. In the src/components we have:

BarGraph
PieChart
Plots: It puts the charts together.
Sent: It creates a card for an individual keyword query
Sents: It contains all the Sent items
Tweets: It shows the actual posts from the social media
Search: It has a input field to take the keyword and a submit button to call the API for the keyword.

AppRemake.jsx renders all the components in the right order and calls the APIs.

Usage

clone the repo
To run the backed

cd backend
pip3 install -r requirements.txt
uvicorn main:app --reload

To run the frontend npm install react cd frontend npm run dev

Challenges

Working on a team and coming to a decision was challenging as we are both competitive.
It was a challenge to run the deep learning model to get sentiment analysis.
Making Web app.