Nightlights Processing Utils

Inspiration

Since I personally live in one of the countries most affected by climate change (Philippines), I'm looking for ways to use data science to mitigate its effects -- especially in typhoon disaster response and recovery.

After building a package for wrangling geospatial datasets, I'm looking for ways to generalize it outside of its intended use cases for poverty mapping.

What it does

This project was originally an attempt to answer a simple question: Can the nightlights data over disaster affected areas before and after the disaster be used to predict the damage over those areas?

Our output was going to be an executable report (with a jupyter notebook behind it so that future researchers can easily replicate our work)

I think I learned how hard it is to work with the night lights data -- I couldn't even begin to search for the images for the my areas of interest.

SO Instead, I started building a set of utilities to help me and others interested in using nightlights data find the relevant images for the dates and areas they are interested in.

Why is this relevant?

Nightlights data has been used as a proxy for economic development and is useful for tracking relevant statistics related to SDG - No Poverty.

I was hoping to see whether it could be useful for typhoon damage assessment (SDG - Climate Action).

How we built it

I used a python package called nbdev that allows me to build the documentation as well as the python package using Jupyter notebooks.

It uses all the standard geospatial libraries, such as geopandas and rasterio.

Challenges we ran into

The biggest challenge to do analysis is simply finding the relevant images in the nightlights dataset for the period and area of interest.

Accomplishments that we're proud of

Being able to build a useful python package that will hopefully be used by others doing research using nightlights data hosted on AWS.

What we learned

A lot of the work for this project involved data preparation, I think we underestimate the effort and value of the data preparation work in order for the "upstream" tasks of building models and presenting cool visualizations to be even possible.

What's next for the nightlights processing utils

I'm hoping to further enhance it to optimize downloads using the COGS features of rasterio. This will further minimize the amount of data needed to be downloaded for a particular area of interest.

And once I have an easy-to-use package to find the relevant nightlights data, I hope I can finally do the original goal of answering the question of whether we can use nightlights data for damage assessment.

Built With

• nbdev
• python