Inspiration
Most people don't realize that over 70% of our oxygen actually comes from the ocean, yet we’ve only managed to monitor about 5% of it effectively. The current methods for tracking environmental damage are often far too slow and rely on massive amounts of manual labor. We were inspired to build a tool that could identify these threats instantly by using the latest breakthroughs in Zero Shot computer vision moving from passive observation to active detection.
How we built it
We built the core detection engine using Python and Streamlit. The brain of the project is a Vision Transformer leveraging the OpenAI CLIP model.
We chose a Zero Shot approach because traditional AI usually requires thousands of specifically labeled images to work, which is a bottleneck for rare environmental data. By using semantic understanding, our model calculates the cosine similarity between image features and text embeddings:
$$ \text{similarity} = \frac{\mathbf{I} \cdot \mathbf{T}}{|\mathbf{I}| |\mathbf{T}|} $$
This allows the model to "reason" through what it sees based on natural language descriptions (like "bleached coral" or "oil spill"), rather than just memorizing pixel patterns.
Challenges we faced
The biggest hurdle was definitely the technical infrastructure. Managing a machine learning environment on a local machine with limited resources is a constant battle. We had to implement a custom virtual environment strategy and optimize the model loading process to ensure the engine would run smoothly during a live demo without crashing the host system. Getting the Vision Transformer to stay performant while maintaining high confidence scores was a significant engineering challenge.
What we learned
This project really showed us how much potential semantic AI has for environmental science. Moving from models that just scan pixels to models that actually understand concepts makes these tools very flexible. We learned that we can build and deploy helpful technology much faster in the field where it's needed most by leveraging pre trained multi modal networks.
What's next for OceanGuard
Moving forward, we want to connect this engine to satellite apis and underwater drone feeds. Our ultimate goal is to create a 24/7 autonomous monitoring network that can protect coral reefs and marine ecosystems around the world in real time.
Log in or sign up for Devpost to join the conversation.