OceanEye dashboard detects ocean trash with YOLOv8, runs RAG via Gemini for insights, and visualizes optimized cleanup routes in real-time.
Cleanup paths are optimized by clustering trash sites and smoothened using Catmull-Rom spline interpolation for boat-friendly routes.
Plot satellite data from Google Earth Engine to visualize debris, clean water zones, and pollution trends on interactive maps.

🌊 OceanEye

Inspiration

Every year, millions of tons of plastic enter our oceans, and over 100,000 marine animals die from entanglement or ingestion of waste. We watched footage of turtles struggling with plastic rings and dolphins swimming through oil streaks, and it struck us:
the ocean is vast, but our cleanup strategies are still manual, slow, and inefficient.

We wanted to build something that could help — not someday, but now.
OceanEye started with a simple goal: use AI to detect, map, and optimize the cleanup of ocean waste while minimizing carbon emissions and protecting marine life.

What it Does

OceanEye analyzes ocean footage in real-time, detecting different types of trash floating on the water surface. It then:

Identifies and classifies waste (plastic bottle, fishing net, bag, etc.)
Maps the geographic coordinates of detected trash clusters
Optimizes the cleanup route to minimize travel distance, fuel, and ecological disruption
Generates detailed environmental insights using generative AI to help researchers and cleanup organizations understand pollution patterns over time

How We Built It

OceanEye is powered by a full AI + simulation + web visualization pipeline:

🧠 Detection Model

Fine-tuned YOLOv8 on custom drone & satellite imagery datasets of ocean debris
Used data augmentation to simulate wave distortion, glare, and turbidity
Performs instance segmentation to outline the exact shapes of waste items

📡 Spatial Mapping & RAG Pipeline

Incorporated Sentinel-2 satellite coastal data and Google Earth Engine (GEE) to detect high-probability garbage accumulation zones and track pollution spread
Detection outputs are pushed into a Retrieval-Augmented Generation (RAG) pipeline using Google Gemini

Gemini analyzes:

Historical pollution correlations
Seasonal tide + wind patterns
Geo-temporal clustering behavior

Gemini then generates cleanup insights, trend reports, and risk assessments

🚤 Route Optimization & Cleanup Simulation

To determine the most efficient cleanup path:

Component	Method / Algorithm	Result
Cluster Sorting	Nearest Neighbor + 2-Opt Heuristic	Groups waste sites into efficient pickup clusters
Path Smoothening	Catmull-Rom Spline Interpolation	Generates smooth, boat-friendly traversal paths

Dynamic pickup route visualization

🌐 Web Platform

Layer	Tech
Frontend Hosting	Netlify
Backend Hosting	Vultr
Frontend Framework	Next.js (App Router) + React + TypeScript
UI Components	Tailwind CSS v4, Radix UI, shadcn/ui
Charts & Visualizations	Recharts + R3F
Backend	Flask + Flask-CORS
AI & ML	PyTorch, YOLOv8, Google Gemini API
Image/Video Processing	OpenCV + Pillow
Satellite Data Analysis	Google Earth Engine (GEE)
Data Storage	File + vector store for RAG context

Challenges We Ran Into

Training the model to distinguish ocean surface glare from actual plastic required extensive augmentation and tuning
Integrating multiple data modalities (video + satellite imagery + geo-coordinate metadata) required custom preprocessing pipelines
Ensuring the cleanup path was both fuel-efficient and eco-safe involved balancing cost functions rather than just minimizing distance
The RAG pipeline required careful prompt + metadata design so Gemini generated useful insights instead of generic summaries
Working with satellite data in GEE required preprocessing for cloud cover, water turbidity, and aligning geospatial coordinates with detected trash

Accomplishments We’re Proud Of

Built a functional ocean waste detection AI system with strong real-world segmentation performance
Created a dynamic cleanup route optimizer with biologically safer navigation paths
Integrated satellite data from GEE for debris monitoring and pollution tracking
Demonstrated how AI can meaningfully contribute to environmental conservation
Successfully integrated detection + simulation + analysis into a single operational dashboard

What We Learned

Fine-tuning models on real-world imagery isn't about getting accuracy — it's about robustness
RAG pipelines require clean, structured context to produce meaningful intelligence
Optimization isn’t purely technical — it must respect ecosystems, wildlife, and ocean currents
The simplest algorithms (like nearest-neighbor) can outperform complex ones when tuned well
Satellite data analysis requires careful preprocessing and integration with real-time AI detection outputs