Inspiration

Our inspiration for this project stemmed from a shared passion for conservation and ecological balance. The monarch butterfly, an iconic species known for its incredible migration across North America, has been facing a sharp decline in recent years. We were struck by the urgency of addressing the decline in pollinators, especially given the monarch’s role in maintaining ecosystems and supporting agriculture. This challenge felt like a natural fit for us, as it combined our data science skills with a cause that we deeply cared about.

Participating in the 2024 Rowdy Datathon at UTSA provided the perfect platform to apply our technical knowledge while contributing to a meaningful cause. We were excited about the opportunity to use data to not only understand but also potentially help mitigate the factors leading to the decline of monarch butterflies.

What it does

Throughout the project, we learned a great deal—not only about the monarch butterflies and the environmental factors affecting them, but also about the technical processes involved in analyzing large and complex datasets. Some of our key takeaways include:

The power of data science in addressing ecological issues: It became clear that data can reveal hidden patterns and correlations that help understand environmental problems. Collaboration is key: As a duo, we learned how to leverage each other’s strengths, share responsibilities, and support each other through challenges. The importance of data cleaning and preprocessing: We faced various data quality issues, from inconsistent formats to missing values, which showed us the critical role that proper data preparation plays in ensuring meaningful analysis. Using APIs for geolocation: One of our most valuable technical lessons was using APIs like Nominatim OpenStreetMap to convert city-level data to county-level insights, a task that was essential for our deeper analysis. Ecological interdependencies: Our analysis underscored how various factors—pesticide use, air quality, and climate change—interact to create complex ecological problems that require interdisciplinary approaches.

How we built it

We built our project around the following steps:

Data Collection: We gathered data from multiple sources, including Journey North for monarch sightings, EPA’s Air Quality System, and USDA’s Pesticide Data Program. This provided us with a wealth of information on environmental factors and monarch migration patterns.

Data Cleaning and Preprocessing: Once we had our datasets, we spent a significant amount of time cleaning and formatting them. This included handling missing values, normalizing date formats, and geocoding city-level data to county-level information using Nominatim OpenStreetMap API.

Exploratory Data Analysis (EDA): We visualized the data to understand migration patterns and identify potential correlations. This involved generating maps of monarch sightings and overlaying them with pesticide use and air quality data to spot trends.

Statistical Analysis: We conducted a statistical analysis to quantify the relationships between the decline in monarch sightings and environmental stressors like pesticide use and poor air quality. We also examined the impact of climate change on migration timing.

Presentation and Reporting: Finally, we prepared our findings for the Datathon, presenting the results in an interactive format using Streamlit. This allowed us to share our insights with a broader audience, making our findings both accessible and actionable.

Challenges we ran into

Like any project, this one came with its fair share of challenges:

Data Granularity: One of the biggest challenges was working with city-level data and needing county-level insights. We had to develop a workflow using the Nominatim API to infer the county based on city and state, which added complexity to the data preprocessing stage.

Large Datasets: Dealing with large datasets from sources like Journey North and EPA required us to optimize our data processing pipeline. There were moments when performance bottlenecks slowed down our progress, but by dividing tasks and streamlining our code, we were able to overcome this.

Data Quality Issues: Incomplete and inconsistent data posed a significant challenge, especially when working with citizen-science data. We had to employ various cleaning techniques to ensure the data was usable for our analysis.

Interpreting Complex Correlations: While we expected to see direct correlations between certain factors and monarch decline, the reality was more nuanced. This led us to iterate several times on our statistical analysis to make sure we were accurately capturing the relationships between variables.

API and LLM Issues: Due to the reliance on the Nominatim API for geocoding city-to-county data, we encountered occasional delays and limitations in the number of API calls we could make. This affected the speed of our data processing. Additionally, when leveraging Large Language Models (LLMs) to assist in some of the automation tasks, we found that their outputs were sometimes inconsistent, requiring us to manually verify and correct results. This added extra time to the project, but also taught us the importance of blending AI tools with human oversight.

Accomplishments that we're proud of

We were able to concur results, though not coming up with the exact output we initially expected. Despite this, we take pride in several key accomplishments:

Successful Data Integration: One of our major achievements was the successful integration of multiple datasets from different sources, such as monarch sightings, air quality, and pesticide use. This allowed us to build a comprehensive analysis framework, even with the complexities involved in normalizing and processing the data.

Geospatial Mapping of Monarch Sightings: We’re proud of our ability to visualize monarch migration patterns at a granular level, converting city-level data to county-level insights. This required us to work with APIs like Nominatim and develop a workflow to ensure accuracy. This geospatial mapping was a major success in revealing regional trends in monarch populations.

Effective Collaboration and Teamwork: Throughout the project, our collaboration was smooth and efficient. We were able to leverage each other’s strengths to tackle different aspects of the project. The balance we achieved in task distribution and problem-solving is something we’re particularly proud of, as it allowed us to work through challenges together.

Gaining Actionable Insights: Though we didn’t achieve the exact outcomes we initially expected, we were still able to draw meaningful correlations between environmental factors and monarch population declines. Our findings related to pesticide use, air quality, and climate effects can contribute to future research and conservation efforts.

Building an Interactive Visualization Tool: Using Streamlit, we developed an interactive tool to present our findings. This accomplishment not only allowed us to showcase our results effectively but also made the analysis accessible to others in an intuitive and engaging format.

In summary, while the results didn’t fully match our expectations, the insights we generated and the skills we developed throughout the process are accomplishments we take great pride in.