Tweet Disasters

• 

  Disaster Tweets Poster

Tweet Disasters

Who:

Kelly Patel (kpatel85), Jialiang Zhou (jzhou88), Jiahao Wei (jwei10)

Introduction:

Build a deep learning model that predicts which tweets from Twitter are about real disasters and which ones aren’t. This is a classification problem with NLP. We came across this idea because it was a Kaggle competition and seemed like a cool thing to try out! Our group will implement two architectures: LSTM and BERT for binary classification and compare the performance.

Related Work:

Are you aware of any, or is there any prior work that you drew on to do your project? There is an article “Predicting Disasters from Tweets Using GloVe Embeddings and BERT Layer Classification” which highlights the use of a BERT Classifier on Tweets. The authors use GloVe Global Vectors for Word Representation embeddings to convert tweets into vectors and then pass the results into a BERT classifier. They achieve about 87% accuracy, which is higher than LSTM, Decision Trees, and Naive Bayes historically.

In this section, also include URLs to any public implementations you find of the paper you’re trying to implement. Please keep this as a “living list”--if you stumble across a new implementation later down the line, add it to this list.

Predicting Disasters from Tweets Using GloVe Embeddings and BERT Layer Classification https://link.springer.com/chapter/10.1007/978-3-030-95502-1_37 BERT Pretraining: https://arxiv.org/pdf/1810.04805.pdf BERT Implementation example: https://www.tensorflow.org/text/tutorials/classify_text_with_bert BERT Disaster Tweets: https://medium.com/analytics-vidhya/bert-for-identifying-disasters-from-tweets-50eeb6844302 BERT Disaster Tweet Walkthrough: https://www.analyticsvidhya.com/blog/2021/12/disaster-tweet-classification-using-bert-neural-network/ LSTM Example: https://www.kaggle.com/code/mariapushkareva/nlp-disaster-tweets-with-glove-and-lstm

Data: What data are you using (if any)?

Provided by Kaggle. 10,000 tweets that were classified by hand. Kaggle has provided 2 csv files, one for training, one for testing. We will split the training set into training, validation, and testing since it is the only labeled set for the purposes of our project. It has the following columns: Id - the unique ID for each tweet Text - the text of the tweet Location - the location the tweet was sent from (may be blank) Keyword - a particular keyword from the tweet (may be blank) Target (TRAIN ONLY) - this denotes whether a tweet is about a real disaster (1) or not (0). The training set is about 990 KB

Methodology: What is the architecture of your model?

We will be implementing 2 architectures. One BERT binary classifier and one custom LSTM classifier. The BERT architecture will come from an existing paper, and the LSTM will be something new! When we have completed both, we will compare the results.

Metrics: What constitutes “success?”

We plan on hyperparameter tuning and running our models on shuffled data. We will be using accuracy, but probably also a ROUGE, BLEU score scheme routine which is often used for NLP. In the papers we have read, it seems most authors use accuracy and precision to rate their models, so we will probably follow suit.

As a base goal, we want to get over 55% with both models so that it seems the architectures are doing better than random guessing. As a target goal, the LSTM model should achieve >=60%, and the BERT model should achieve between 75-85%. As a stretch, it would be great if LSTM achieves over 70% and the BERT model achieves close to 90% accuracy.

Ethics:

Twitter has become an important channel for communication in times of crisis. With the proliferation of smartphones and mobile technology, people can announce emergencies they’re observing in real time. Because of this, agencies and media are interested in monitoring Twitter for quick reporting. Unfortunately, sometimes it is not always clear when a tweet is about a real disaster or something more trivial. We are unsure how the data was collected and whether the people who created these tweets were informed that their tweets are being used for this competition. This brings up questions and concerns about consent.

Division of labor:

Kelly has a personal GPU, so she will do the computationally intensive parts of the project like data preprocessing and hyperparameter tuning. Jiahao will be implementing the custom LSTM model Jialiang will be implementing the BERT model from a paper All three members will contribute to the papers, presentation, and other deliverables.

Built With

• geforce
• hub
• python
• tensorflow