Analyzing Tweet Sentiment

Title

Who

Anna Novatney (anovatne), Darren Fang (dfang8), Hunter Adrian (hadrian), Jay Lin (jlin142)

Poster

https://docs.google.com/presentation/d/1exH4ry6qi4XXASigc15ZVla8TLugazkkctZxPgSajCE/edit?usp=sharing

Final Reflection & Github

https://docs.google.com/document/d/1rMBaOzO-Fs7aE85spwfH_c-JI7RP9aOa0L_VcukGJKY/edit

Check In 2

Introduction

The paper's objective is to develop a deep learning model that can predict the sentiment polarity of tweet replies. The paper highlights the importance of understanding the sentiment of tweet replies as it can provide insights into how the original tweet is perceived by the audience and can be used to analyze the effectiveness of communication strategies. We chose this paper because it addresses a practical problem that is relevant in today's social media-driven world—Twitter has become an important platform in terms of information sharing. The problem falls under the category of supervised learning, specifically, classification. The objective is to classify the sentiment polarity of tweet replies into one of three categories: positive, negative, or neutral. The paper proposes a model that is trained on a dataset of labeled tweet replies, where the sentiment polarity of each reply is labeled as positive, negative, or neutral. The model is then used to predict the sentiment polarity of new, unseen tweet replies.

Related Work

Our project will be a translation of the paper How Will Your Tweet Be Received? Predicting the Sentiment Polarity of Tweet Replies from PyTorch to TensorFlow. Here is the link to the github: https://github.com/tayebiarasteh/retweet. There have been quite a few papers that deal with deep learning and sentiment analysis of tweets. Another paper that is related is "Sentiment analysis of Twitter data" by Hemant Kumar Aggarwal and R. K. Aggarwal. A link to this paper can be found here: https://www.researchgate.net/publication/312216311_Efficient_Twitter_Sentiment_Classification_using_Subjective_Distant_Supervision. This paper explores the task of sentiment analysis on Twitter data, specifically focusing on the classification of tweets into positive, negative, and neutral categories just like the paper we are focusing on now, except instead of focusing on the sentiment of the replies, the focus is on the sentiment of the tweet itself. This paper uses a combination of both lexicon-based and machine learning approaches. They first preprocess the tweets by removing stop words and tokenizing the remaining words. The preprocessing in this paper is incredibly interesting. I really liked how they dove into the differences you may find between formal writing and a tweet. This included purposeful misspellings for effect and different punctuation. They then use a lexicon-based approach to classify the tweets based on the presence of positive and negative words. Finally, they use a Support Vector Machine (SVM) classifier to classify the remaining tweets that could not be classified by the lexicon-based approach. This paper also had a whole section on related works. There is no shortage of work on sentiment analysis of tweets.

Data

We will be using the dataset RETWEET which can be found in Kaggle at https://www.kaggle.com/datasets/soroosharasteh/retweet. This dataset was created by the same group that wrote the paper that we are reimplementing. This dataset contains tweets and the sentiment of their replies. The training dataset contains a total of 34,953 tweets and the testing dataset contains a total of 1,519 tweets. As we are using the same dataset used for the paper, we do not anticipate significant preprocessing.

Methodology

The model's architecture is based on a recurrent neural network featuring two Bi-directional Long-Short Term Memory Units (BiLSTM) as its main components. The network includes an embedding layer, two BiLSTM layers with hidden and cell states of length 256, dropout layers after each BiLSTM, and a fully connected layer followed by a SoftMax function for classification. We are training the model using a dataset consisting of labeled tweets with positive, negative, or neutral sentiment. The model is optimized with the Adam optimizer, and the loss function is weighted cross-entropy with inverted class frequencies of the training data to handle dataset imbalance. The most challenging aspects of implementing this model are likely to be collecting and preparing the large amount of data required for training and validation, as well as fine-tuning the model's hyperparameters to achieve optimal performance.

Metrics

Experiment-wise, we plan to use the same RETWEET dataset provided by the original authors to test if our translation from PyTorch to Tensorflow is accurate. Additionally, the notion of accuracy applies for our project. Since we are using the same dataset and implementing the same architecture as the original authors, we are expecting very similar accuracy results. In addition to an accuracy metric, we will also be using the F1, Recall, and Precision metrics. The authors of the paper were hoping to expand basic Tweet sentiment analysis to Tweet reply sentiment analysis. The authors want to find the answer to the question of “Given a tweet, what will be the sentiment of its replies?” The authors also expanded upon this to find the correlation between a Tweet’s sentiment and its replies’ sentiment. To quantify the results, the authors used metrics such as accuracy, F1, Recall, and Precision. The results would be compared to a baseline of assuming that a Tweet’s reply sentiment matches the original Tweet’s sentiment. What the paper showed was that applying a neural network improves the metrics. Our base goal would be to translate the paper architecture into Tensorflow. Our target goal is to match the metric results of the paper using the dataset provided. A stretch goal would be to extend the existing architecture and test it on different Tweet sentiment datasets.

Ethics

There are several broader societal issues that are relevant to sentiment analysis of tweet replies and social media data in general. First is privacy: social media platforms collect vast amounts of personal data from their users, including their posts, comments, and other interactions. Sentiment analysis models can use this data to make predictions about individuals' emotions and opinions, which could potentially be used to target them with personalized advertising or other forms of manipulation. This raises concerns about data privacy and the ethical use of personal data. Second is free speech: social media platforms are often used as forums for public debate and discussion, and sentiment analysis can be used to track and analyze the opinions and emotions expressed by different groups. However, there exist concerns about the potential for sentiment analysis to be used to suppress free speech or individuals based on their opinions or beliefs.

Deep learning is a good approach for predicting the sentiment polarity of tweet replies because it allows the model to automatically learn complex patterns and representations from the large amounts of textual data available in social media platforms like Twitter. Traditional machine learning approaches rely on manually engineered features, which can be time-consuming and may not capture all relevant information. In contrast, deep learning models can automatically extract relevant features from raw text, such as word embeddings that capture the semantic relationships between words. This allows the model to effectively capture the nuances and complexities of language, which is especially important in social media data where there is often slang, abbreviations, and other forms of informal language. Additionally, deep learning models can be trained on large amounts of data, which is crucial in sentiment analysis since the meaning and usage of language can vary widely across different contexts and user groups. By training on large amounts of data, deep learning models can learn to generalize better and make more accurate predictions on new and unseen data.

Division of Labor

Jay, Hunter, Darren - Code Anna - Preprocessing

Check In 3

Introduction

The paper's objective is to develop a deep learning model that can predict the sentiment polarity of tweet replies. The paper highlights the importance of understanding the sentiment of tweet replies as it can provide insights into how the original tweet is perceived by the audience and can be used to analyze the effectiveness of communication strategies. We chose this paper because it addresses a practical problem that is relevant in today's social media-driven world—Twitter has become an important platform in terms of information sharing. The problem falls under the category of supervised learning, specifically, classification. The objective is to classify the sentiment polarity of tweet replies into one of three categories: positive, negative, or neutral. The paper proposes a model that is trained on a dataset of labeled tweet replies, where the sentiment polarity of each reply is labeled as positive, negative, or neutral. The model is then used to predict the sentiment polarity of new, unseen tweet replies.

Challenges

One of the hardest parts of the project we've encountered so far has been integrating the data with the model. We faced issues while converting the data processing method used in the original research paper from PyTorch to TensorFlow. Data processing in this framework is quite different from what we have done previously in the homeworks for this class so that has been a challenge. Another, less technical, challenge has been finding time for us to be able to work together. It is a really busy time of year, so sometimes it is hard to fit everything in!

Insights

We have successfully implemented both the biLSTM and CNN architectures, transitioning from PyTorch to TensorFlow. We also managed to integrate a new data source into our project which will hopefully make the data processing step easy. We do not really have any results at the moment for our project, but we are definitely making progress!

Plan

Currently, we are a little behind schedule in our project timeline. Our main focus is on implementing a new dataset, which should help us get back on track once we successfully integrate it. To catch up, we need to dedicate more time to testing and training the model, as well as producing visualizations that can help us better understand its performance. We also know that the deadline for printing out our poster is coming up soon - by Thursday. We also need to schedule time so we can make enough progress on our model in order to create a presentation.