Shoulder Watch

Shoulder Watch is AI that watches Georgia’s 3,700 public traffic cameras, spots stalled cars before they’re reported, and texts drivers via iMessage so they can move over safely.

Comment

Inspiration

A car stalled on the shoulder of an Atlanta interstate is one of the most dangerous places to be on the road, for the driver and for everyone coming up behind them. Usually the first warning anyone gets is brake lights. Georgia DOT already runs hundreds of public traffic cameras, but no one can watch all of them at once. We wanted software that watches them for you, then reaches drivers in the app they already use: iMessage.

What it does

Shoulder Watch watches 34 public GDOT interstate cameras around metro Atlanta and spots vehicles stopped on the shoulder or stalled in a lane.

  • Live dashboard: a camera wall with vehicle boxes, a live map with traffic levels and pulsing stall markers, and a detections log with a snapshot timeline for every flagged vehicle.
  • iMessage agent (Photon Spectrum): text it like a friend:
    • how's traffic on I-85? → traffic level by area plus live camera photos
    • any stalls? → every active stalled vehicle, with photos
    • watch I-75 for 2 hours → get an alert like "⚠️ Stationary vehicle, I-85 S at Clairmont Rd, stopped 6 min. Move over if you can."
    • stop → unsubscribes you and deletes your number
  • AI double-check: Gemini reviews each flagged frame and returns a strict structured verdict.

How we built it

  • See: a COCO-pretrained YOLOv8s model runs locally through ONNX Runtime on CPU, with no cloud vision. It detects cars, trucks, buses, and motorcycles on every fresh frame.
  • Understand: a pure, unit-tested tracker matches vehicles across frames by position. A vehicle that stays in the same spot for 3+ fresh frames while other traffic keeps moving is a stall candidate. If most vehicles in view are stopped, that's congestion, not a stall.
  • Verify: a pixel-level arrival check compares the vehicle's area against earlier frames, so signs, barriers, and lane markings that "never move" get rejected. Gemini 2.5 Flash then gives a structured verdict on each flag.
  • React: a Photon Spectrum (spectrum-ts) agent handles questions, subscriptions, and rate-limited alerts.
  • Stack: one TypeScript codebase. A Node worker does polling, detection, and the agent, and a Next.js 15 / React 19 dashboard shows the results. Data lives in plain JSON and PNG files instead of a database, so there's nothing to break at a venue.
  • Replay mode: recorded real detections run through the same pipeline and are always clearly labeled REPLAY, so the demo still works if the Wi-Fi doesn't.

Challenges we ran into

  • Feeds refresh slowly: we expected new frames every minute, but at night they refreshed only every ~7–12 minutes. We rebuilt the tracker around fresh frames and frozen-feed detection (identical image hashes), so a stuck feed can never cause an alert.
  • Things that never move: wide lane markings on I-20 scored as "car" at 0.5–0.6 confidence. Because they never move, they looked like perfect stalls. The arrival check fixed this.
  • Parked cars: cars in a lot beside I-85 got flagged as stalls. We added per-camera ignore zones and a "long-parked" relabel.
  • Camera operators pan and zoom: we detect scene changes with thumbnail diffs and reset tracking.
  • Night: dark, glare-heavy frames make the model miss vehicles, so the dashboard labels night tiles "low reliability" instead of pretending.

Accomplishments that we're proud of

  • A real end-to-end system: public cameras → local computer vision → stall logic → AI check → an alert on your phone in iMessage.
  • It tells a stall from a traffic jam from a camera pan from a frozen feed.
  • Honest by design: stale data is never shown as live, replay data is always labeled, and unverified detections never send alerts.
  • Privacy first: only public low-res images, no plate reading, no identifying people, and phone numbers deleted on stop.
  • A unit-tested core: tracker, arrival check, command parsing, traffic levels, and rate limiting.

What we learned

  • Real-world data is messier than any dataset: slow feeds, glare, panning cameras, things that look like cars.
  • A stall detector is mostly about what not to flag.
  • Meeting people where they already are (iMessage) matters more than another dashboard.

What's next for Shoulder Watch

  • 511GA API integration for exact camera coordinates and an automatic comparison against official incidents, to measure how much earlier we spot stalls.
  • More cameras (GA-400 and beyond the core metro).
  • A model that works better at night.
  • Alerts routed to first responders and GDOT HERO units.
  • Location-based alerts: text a pin and get the cameras near you.

Built With

Share this project:

Updates

Submission history