Inspiration

This project is a continuation of a research project I'm doing at MIT, where the goal is to detect natural disaster images "in the wild". This is useful because social media blows up during and after natural disasters, and right now there is no way to monitor these data streams effectively. I created a dataset and classification model before the hackathon, but I improved the model and demonstrated it's effectiveness on images in this hackathon.

What I came in with:

  • Dataset and computer vision model.
  • Some viewer code I wrote previously to display images with their latitude and longitude on a map.

What I did here:

  • Downloaded many images and ran evaluation scripts.
  • Showed effectiveness for ~1M images on Flickr, wildfire images from Twitter, ground-level flood images (with Twitter data I have from a collaborator but not publicly available), and a weather monitoring histogram for a region. I tried showing results for earthquake filtering, but this was more noisy.
  • Scraping, image downloading, planning, and plotting took the majority of time.

What it does

Runs classification on images to predict whether a natural disaster exists or not. This can be used "in the wild" and during natural disasters with social media images.

How I built it

I came in with a ML model, and I worked with OpenCV, Matplotlib, Plotly, and Twitter/Instagram APIs this hackathon to show the model's effectiveness.

Challenges I ran into

N/A for the hackathon, but a general issue is handling "false positive" images. This is an ongoing challenge, and I did some modifications to the network during my hack.

Accomplishments that I'm proud of

Showing that the model works well for Instagram images, Flickr images, floods, fires, and more. I really wanted to showcase a suite of applications for my hack, and I think I almost finished the whole list. I didn't have as much time to run correlation statistics with existing datasets.

What I learned

I learned that API limits are restrictive and how to parallelize my work when waiting for time-consuming scripts (when downloading and dealing with over a million images) to run.

What's next for Detecting Ground-Level Natural Disaster Images "In The Wild"

This work will go into a paper I'm planning to submit to a conference soon. I'm glad I could showcase the effectiveness of my model during this hackathon through web scraping of images and filtering with many experiments.

Share this project:

Updates