Social media, most notably Twitter, has played a key role in the distribution of information during earthquakes. People use Twitter to alert and inform other citizens, and this is where media outlets source much of their early-stage information. Yet, in many less developed countries across the world, this source of information is underutilized. Emergency response systems are often alerted hours or days after the earthquake takes place.
Earthquake Watch is a real-time worldwide earthquake monitoring platform that does the following:
- Mines real-time data from Twitter with Twitter's API and web scraper
- Extracts the relevant information through Natural Language Processing
- Predicts the locations and relative magnitudes of earthquakes through Latent Dirichlet Allocation and a LSTM recurrent neural network.
It can be used as a tool to allow the appropriate respondents, such as disaster relief agencies and humanitarian organizations, to more quickly act upon earthquakes across the world.
Challenges I ran into
No matter how much relevant information there is on Twitter, there is always an equal amount, if not more, of irrelevant thoughts, advertisements, and even misinformation. One of the major challenges of this project was to process these tweets and extract the most relevant data through Natural Language Processing and learning models.
How I built it
Earthquake Watch is entirely built on Open Source software. I gathered data with Twitter's API, used TensorFlow for modeling, and nltk and gensim for Natural Language Processing inference. I served up my application with Flask.
As well as improving the accuracy of Earthquake Watch's predictions, I plan to migrate the application to AWS infrastructure in order to improve performance and scalability.