ShipTrace

Communication test between arduino uno, esp32, and raspberry pi 3b
Searchlight motor mounting and wiring
Testing computer vision with a small orange bottle in a field of beige

Inspiration

When someone falls off a ship, the hardest part of the rescue is the first 60 seconds: crews lose sight of the person almost immediately, and every second of searching lowers their chance of survival. Modern vessels are starting to use automated man-overboard detection to solve this, but that raised a question we couldn't stop thinking about: these are networked safety systems, and a networked safety system is an attack surface. What happens when someone sends it a malicious command mid-rescue? We wanted to build a rescue device that not only finds a person in the water, but refuses to be hijacked while doing it.

What it does

SHIPTRACE is a ship-mounted rescue spotter. An overhead camera scans the water for a high-visibility target (a person in an orange or neon-green life jacket), computes the bearing to them, and automatically turns a servo-mounted searchlight to point rescuers in the right direction.

The security layer is the heart of it. Every steering command is authenticated with HMAC-SHA256 and a sequence number. When an attacker tries to take control — with a forged command, a replayed one, or a flood of them — the device detects it and triggers a physical response: a relay cuts power to the actuator, a red light comes on, and the system latches into a SAFE MODE that can only be cleared by a human pressing a button. After the fact, the captured attack traffic is turned into a readable incident report.

How we built it

Vision (Python + OpenCV): HSV color detection on a Logitech C270 feed finds the largest target blob, computes a servo bearing, and emits it as rate-limited JSON. Ground control (Python): signs each bearing into a command (v1|seq|cmd|arg|hmac) and sends it over Wi-Fi via UDP, with a deadband and heartbeat so its own traffic never looks like an attack. Rescue beacon (ESP32 / C++): verifies every packet through a strict pipeline — rate → format → HMAC → replay → command → bounds — drives the servo, and controls the relay through a transistor so that an unpowered GPIO always means a de-energized actuator (fail-safe by design). Attack + forensics (Python + Scapy): a red-team harness that fires each attack class at our own device, a live security console, and a packet-forensics stage that feeds structured evidence to an LLM for the incident summary.

The safety decision is fully deterministic; AI is used only to explain incidents, never inside the control loop.

Challenges we ran into

Hardware roulette. Our original plan used a stepper motor and a floating hull, but the parts we could actually source pushed us to pivot to an ESP32 beacon and a servo-driven spotter — a better, more reliable demo under a 36-hour clock. Not tripping our own defenses. Our first ground-control code sent commands fast enough to trigger the beacon's own flood detector. We had to add rate limiting, a deadband, and a heartbeat so legitimate traffic stays comfortably under the threshold. Making replay protection honest. Restarting a component can't be allowed to look like a replay, so we switched sequence numbers to millisecond-epoch timestamps that always increase. Getting the HMAC to match across languages. Aligning the exact signed byte string between Python and the ESP32's mbedTLS implementation took careful test vectors.

Accomplishments that we're proud of

A security response that is physical, not cosmetic — a relay that actually kills power, not just a warning on a screen. Defense in depth that we verified attack-by-attack: unsigned, forged, replayed, tampered, and flooded commands each trip a distinct, correct rejection. A design that fails safe on boot, crash, or a cut wire, and that can only be recovered by a human — an attacker can't unlock it over the network. A clean split between deterministic safety logic and AI-assisted analysis, so the intelligent part never endangers the safety-critical part.

What we learned

How to design an authenticated command protocol from scratch — HMAC, replay protection, and rate limiting — and how each maps to a real attack. Why safety-critical systems keep the "decide" and "explain" jobs separate, and why hardware enforcement beats software flags. The practical realities of embedded logic levels, transistor relay driving, and fail-safe wiring. That under a hard deadline, the reliable version of an idea beats the ambitious one.

What's next for ShipTrace

Per-device keys and secure storage (ESP32 flash encryption and secure boot) instead of a shared demo key. Persisting the sequence counter to flash so replay protection survives a reboot. Real localization — swapping the tabletop bearing model for GPS and thermal/radar sensing used in actual maritime MOB systems. A hardened forensics pipeline that streams incidents to shore in real time.