Orion & Diana

Autonomous OSINT agent detecting oil spills faster than any human...

Comment

Inspiration

In 2024, I was on my national team for the world's premier robotics tournament, World Robotics Olympiad (WRO). Our challenge then was building a robot that was capable of autonomously cleaning up oil spills.

The real bottleneck wasn't cleanup. It was detection. Oil spills happen across millions of square kilometers of ocean every year. The Deepwater Horizon disaster killed 11 workers and dumped 4.9 million barrels into the Gulf of Mexico before responders even knew the scale. The Exxon Valdez took days to be fully understood. Countless smaller spills go unnoticed or are discovered weeks after the fact, when environmental damage is already done.

The detection problem today is stuck in the 1980s. Environmental agencies rely on:

  • Manual coastguard patrols (expensive, limited coverage)
  • Citizen reports and news media (delayed, inconsistent)
  • Periodic satellite imagery review (hours to days of latency)
  • Reactive tasking: someone has to decide to look at a specific location at a specific time

Meanwhile, the technology exists. Sentinel-1 SAR and Sentinel-2 optical satellites image the entire Earth every few days. Machine learning can detect oil slicks with 99% accuracy. Natural language processing can extract location coordinates from news articles in seconds. But integrating these pieces into a real-time system requires end-to-end automation that nobody has built for the general public.

What it does

Orion is a real-time marine oil spill detection platform that combines automated OSINT with live satellite imagery. An autonomous AI agent named Diana continuously scrapes maritime news, extracts location coordinates, and dispatches Sentinel-1 SAR and Sentinel-2 optical satellite scans via Google Earth Engine. The pipeline classifies all 27 tiles in a 3x3 spatial grid using a ResNet model trained on oil slick signatures, combines SAR and optical verdicts, and surfaces results in a live dashboard. The system is honest about uncertainty: high confidence in one sensor but low in another returns "inconclusive," not a false positive.

How I built it

The architecture has three layers:

  1. Diana (Jac-language agentic OSINT): scrapes maritime news, geocodes articles using geonames, and outputs scan jobs as JSON. Runs in a demo loop using curated historical articles. Thinking agent was a Qwen 3 powered by Featherless, in a 4-agent-parallel framework.

  2. Orion backend (Python): receives coordinates and date windows, orchestrates parallel GEE fetches for 9 grid cells across both SAR and RGB, runs dual-sensor ResNet classifiers, and writes manifests to disk. Thinking agent was a Qwen 3 powered by Featherless, in a 4-agent-parallel framework.

  3. earth-watchtower frontend (React + TypeScript, ideated with Lovable and then modified by me): displays the anomaly feed with severity indicators, an inspection panel showing the winning tile and full 3x3 gallery, inline tile renaming, and a dossier narrated by an LLM.

Challenges I ran into

  1. Satellite revisit gaps were a big thing and broke simple detection. I discovered this problem in existing literature on maritime monitoring and solved it using a spatial grid approach: instead of asking "is there oil at this exact point," I made the model fetch a 3x3 grid of cells around the target and classify all 27 tiles in parallel. This increases the probability of capturing the slick geometry even if the satellite pass is slightly offset from the nominal revisit time. I also implemented a model where if the tile concerned has land in it, it is ignored and furthermore shifted to find just the ocean (even land in a satellite can fool the model into thinking it's oil!).

  2. Dual-sensor consensus: RGB classifiers often return high confidence while SAR is unavailable (inland or cloudy). Solved by using max(S1, S2) consensus and marking cases as inconclusive when one sensor is missing, rather than false-confidence blending.

  3. Concurrent GEE calls: fetching 27 tiles sequentially took too long. Solved by parallelizing with ThreadPoolExecutor (4 workers).

  4. Windows shell context: mixed Bash and PowerShell environments caused path resolution failures. Solved by using dedicated tools (Glob, Read, Edit) for filesystem operations.

Accomplishments that we're proud of

  • End-to-end OSINT + orbital pipeline with zero human in the loop (Diana to dashboard).
  • Dual-sensor consensus that refuses to hallucinate confidence when one source is missing.
  • Honest uncertainty labeling (inconclusive verdicts for Pemex and Cold Lake show the system's integrity).
  • Live React dashboard with inline rename, tile inspection, and dossier narration.
  • The whole stack deployed and working in a single hackathon.

What I learned

  1. Satellite revisit gaps are real and break naive spill detection. The solutions we found were in literature: grid solving where we created a grid based system and avoided lanmd

  2. LLM narration (Diana's dossier writing) is a UX win: users trust a system more when it explains its reasoning in plain English, even if the reasoning reveals uncertainty (+ allowed us to develop the reasoning in the backend :D)

  3. Jac is a great language for agent manipulation: before I would have to manage context, test modules to find what broke, make a detailed control flow (and for Diana this would be a big thing)

  4. Dual-sensor classification is standard in satellite meteorology (radar + optical with both Sentinel 1 and 2 used in this project) and should be standard for spill detection too. I first used Radar and kept failing before realising Sentinel-2 had better RGB models that worked better.

What's next for Orion

  1. Technically, this isn't just an Oil Spill model but an end-to-end OSINT that could be modified for defense, intelligence and more climate impact metrics like polar ice caps, deforestation in Brazil with justa few tweaks of satellite requests. Would be interesting to explore more fun ways to experiment with this model.

  2. Integrate with MARPOL reporting and official incident databases so operators can cross-reference detected slicks against filed incidents.

  3. Real-time alerting: Slack/email notifications to environmental NGOs and maritime authorities the moment a high-confidence spill is detected.

  4. Cost optimization: Google Earth Engine billing scales with fetch volume. Consider caching tiles in a local PostGIS database for frequently-monitored regions.

  5. Extend Diana to scrape beyond English-language maritime press (Arabic, Mandarin, Russian sources for Persian Gulf, South China Sea, Arctic).

  6. Actually find a way to get Diana to run for a month continuously to see what it will find.

Built With

Share this project:

Updates