Star Trek Web Scraper Visualization

Visualizing data gleaned from scraping the Star Trek wiki, Memory Alpha. Specific focus on the breakdown of planets by habitability and officers by rank.

Comment

Inspiration

Both contributors are avid Star Trek fans and wanted to learn how to use a web scraping utility.

What it does

Scrapes the Star Trek Wiki pages listing canon planets to determine what percentage of planets are habitable, and what the distribution of officers by rank is.

How we built it

Built Python scripts that use Beautiful Soup (chosen for its simplicity, we do not have to scrape any dynamically generated pages) to create CSVs from Memory Alpha (the Star Trek wiki). Analyze and visualize results in a Jupyter Notebook using Pandas and MatPlotLib.

Challenges we ran into

Learning how to use Beautiful Soup. Navigating several pages for data (planets were categorized by letter, and officers by the century they were active). Also encountered issues with SSL certifications when trying to open URLs in Python script.

Accomplishments that we're proud of/lessons learned

  • Learning a new skill.
  • Executing a simple project with thought-out plan effectively and with little stress.

Built With

Share this project:

Updates