orcast

Durable forecasting of orca encounters and what conditions predict where they might appear and when. Immersive workbench for behavioral analysis data annotation and 3D reconstruction of Orca behavior.

Comment

Inspiration

The Southern Resident killer whales of the Salish Sea are endangered, and the people who care about them watch from shore, from kayaks, and from research vessels. Most wildlife forecasts show a confident map that hides how thin the underlying evidence is. For an animal this fragile, a map that looks certain when it is not does real harm to trust.

The goal was the reverse, a forecast that shows the evidence it actually has, declines to oversell, and traces every number back to its source. A second idea came from coursework on human and AI interaction, the notion that machine intelligence is most useful when it detects a person's intent, grounds that intent in a real domain, and hands back tools the person can act on, in a loop that improves with use.

What it does

orcast is a two-sided loop around Salish Sea killer whales. Encounter forecasting is the grounding layer that both sides stand on, not the end product.

A side, the visitor console. You tell the console what you are doing, for example planning a kayak trip. It reads that intent, grounds it in the forecast, and opens the planning surfaces you act on. The map flies to the place, the fitness gates and the safe tide and current window come up, and every panel is built on the same forecast. A sighting check separates how likely an encounter was from whether what you saw was an orca. Field notes stay private in a per-user journal until you publish, and shore reports enter a moderation queue where a signed-in reviewer approves before any low-weight attribution.

B side, the research workbench. On /workbench you arrive at the Orcasound Lab hydrophone in Haro Strait, rebuilt in three dimensions. You scrub a real archived recording and the whole scene follows, because one clip and one clock drive the in-browser spectrogram, the presence estimate, and the reenactment together. Early in the clip the model estimates no call and says so. Later it estimates presence, and only then does a reenactment appear, driven by measured Southern Resident tag motion.

Earned confidence. The forecast is always shown, but only at the confidence its gates have earned. When the displayed value is 0 percent, that is the true answer, and the app explains why.

The loop serves three parties at once, the tourists who visit, the researchers who study whale behavior, and the whales themselves.

How it was built

Frontend. Next.js on Vercel, with a 3D Salish Sea twin rendered in WebGL. The console is an intent-to-surfaces planner that decides which panels to open before it narrates.

Backend. A FastAPI service on AWS App Runner, with DynamoDB as the system of record across nine on-demand tables, AWS Step Functions for orchestration, and Amazon Bedrock (Claude Haiku) for sighting narration grounded only in live gate and provenance context.

Forecasting. A negative-binomial encounter model with a battery of fitness gates and published integrity conditions. The response is modeled as a negative binomial, Y ~ NegBinom(μ, θ) with log μ = Xβ. A promotion step and a confidence gate govern what is displayed, so the shown confidence is capped when the held-out deviance skill is not positive. Displayed confidence equals the model confidence c when the held-out deviance skill D_skill is positive and the cell is promoted, and 0 otherwise.

Acoustics. A Perch 2.0 audio embedding feeds an ecotype classification head, evaluated against a held-out DCLDE set, and an in-browser short-time Fourier transform paints the spectrogram from the real audio.

Real-GPU verification. Headless browsers fall back to software rendering, which hides GPU bugs. A Tesla T4 capture host, driven over AWS Systems Manager, confirmed the real renderer string before any frame was trusted.

Challenges encountered

  • Showing 0 percent confidence without making the app look broken. The fix was to make the gate the product and explain the promotion logic on screen.
  • A GLSL ES reserved word, interface, compiled fine in software rendering but crashed the ocean shader on real GPUs. It surfaced only on the Tesla T4, where the symbol was renamed.
  • Animated scenes made structural-similarity parity tests non-deterministic, so a frozen-frame capture hook was added to get a stable comparison.
  • The Tesla T4 is a server-class card, not a client laptop, so its frame times are an upper bound. The client-tier frame-time budget was measured against 30 frames per second, and the server number was not quoted as the result.
  • AWS plumbing took real debugging. IAM permissions for the deploy user, a backend image that was missing a bundled data file and returned 404s until it was packaged, and rotating the shared agent key for the authenticated proxy.

Accomplishments

  • A forecast that displays its real confidence, including a true 0 percent, and traces every cell back to kernels, gate verdicts, and a nearby sample.
  • A research workbench where the spectrogram, the presence estimate, and the reenactment all respond to a single scrub, with the trajectory driven by measured tag motion and the acoustic estimate only gating whether the reenactment spawns.
  • A measured result on claim grounding. The surface-planner step-log brought the unsupported scientific-claim rate to 0 percent in a live audit against the Gemini API, where maps grounding alone left 91 percent of orca-science claims uncited. This is a claim-citation result, not a forecast-accuracy result.
  • A nine-table DynamoDB system of record deployed behind App Runner, with a real-GPU verification pipeline backing every visual claim.

Lessons learned

  • Transparency can be a product feature. Gates that publish their integrity conditions earn more trust than a confident map that hides them.
  • The intent-then-grounding loop is the useful shape for a research tool. Detect what the person wants, ground it in the domain, and return surfaces they act on, then let their activity sharpen the next turn.
  • Preparing the surfaces before narrating, and not narrating first, is what drove the unsupported-claim rate down.
  • Verify on the hardware you ship against. Software rendering hid a shader crash that only a real GPU surfaced.
  • Labeling discipline matters. Measured and modeled are named on every surface, and nothing implies a shipped capability the build does not have.

What's next for orcast

  • Extend the workbench toward dtag modeling replay, with the tag data kept partnership-gated.
  • Widen the station coverage across Puget Sound once the load cost of the larger 3D extent is justified.
  • Build the AI orchestration console that runs the routines across both sides for the shared benefit of the three parties. This layer is direction, chartered and not yet shipped.
  • Strengthen the acoustic heads and resolve which call-type classifier can ship versus stay diagnostic.
  • Add effort correction to community sightings so attribution weight reflects observation bias.

Built With

Share this project:

Updates