earth-cool

Earth-cool is a dynamic web-deployable satellite image viewer with a minimalistic UI. It continuously fetches new image data, and supports several satellites and data overlays.

Comment 2

Inspiration

We have always been inspired by wide, sweeping vistas of our planet. It's hard to top just how small they make us feel - to see all of humanity's contributions reduced to something so insignificant. It's awe-inspiring. But after some research, we discovered sites putting beauty above all else with live shots of Earth are very hard to come by. Clutter is a consistent theme, but even more pervasive is subpar performance. We saw space for significant improvement.

What it does

Cool-Earth maintains a small inventory of processed data files across two NOAA satellites. A dynamic web frontend displays the latest image of Earth by default, and allows for swapping between data type, satellite, and zoom levels. A slider allows for smooth scrolling between the last 12 hours of satellite imagery, and supports animations with variable frame time.

How we built it

NOAA maintains an AWS server for each distinct satellite. Our Python backend uses boto3 to access each server (selection is dependent on parameter) and parse a given time object to directory structure. Time gradually increments from a start point and continues either to a fixed stop point or all newer files have been downloaded.

Raw data files are in the netCDF4 format and dissected into bands with the python library of the same name. Each band is actually a 3D array, which we process and stack through matplotlib. These are saved to varied directories depending on type and zoom level.

Our frontend was built from the ground up. We started by writing scripts in Javascript, with only a very basic shell of what the user interface would look like in the beginning. As we built the necessary functions we needed for the frontend, we gradually added more interactive features to the website that were both essential and cool. In the final stretch, we polished the CSS, adding a clean and tidy finish to our project.

Challenges we ran into

Backend

  • Data structure and credentialing in AWS server
  • Finding image time from filename
  • Memory leak issues and memory usage while processing raw data
  • Properly ordering files as they are downloaded

Frontend

  • Centering images of variable sizing
  • Fading a dynamic wrapper of images
  • Locally caching images to be loaded, client-side
  • Handling file I/O without external interface
  • Optimized exclusively for webkit browsers

Accomplishments that we're proud of

  • Minimalistic, to-the-point UI with unfussy user experience
  • Snappy frontend performance and efficiency
  • Automated file fetching and processing system
  • Processing a single raw data file to multi-band images
  • Maintenance of fixed size image backlog

What we learned

In short, frontend is hard and matplotlib is terribly unoptimized.

What's next for earth-cool

Our larger goal is to expand support to include all satellites including Himawari-8 and Meteosat to provide a live shot from every angle of the Earth. We're aiming to host a server and run the update script through a cron job to provide hands-free maintenance of our system.

Share this project:

Updates