Dockhand

• 

  Supplier scorecard over 30 days: on-time %, short rate, missed orders. Flags the worst supplier. (Includes labeled simulated data.)

• 

  Today's delivery ledger: each drop-off logged with a camera snapshot, box count, and whether it matched an expected order.

• 

  Discrepancies with evidence and a one-click dispute summary for the supplier. (Simulated demo history, clearly labeled.)

• 

  Simulated Alexa+ voice assistant answering from the Dockhand MCP server. Right: the live MCP tool calls it made.

• 

  "Did the US Foods delivery come today?" The assistant calls check_delivery over MCP and returns a card with snapshot evidence.

Inspiration

Small businesses like cafés and restaurants get supplier deliveries several times a week. Drivers drop boxes at the back door, nobody has time to count them, and short or missing orders are discovered days later with no proof to dispute them. Many of these businesses already have a Ring camera pointed at that door. I wanted to see whether that camera could do the receiving job, not just security.

What it does

Dockhand turns a Ring camera into an automatic receiving clerk. When a delivery happens, it:

1. Captures live video from the Ring camera and picks the clearest frames
2. Uses Amazon Nova Pro on Bedrock to read the scene: carrier, box count, visible label text
3. Logs the delivery with a snapshot as evidence
4. Matches it against the orders the business expected, and flags short, over, missing, and unexpected deliveries
5. Produces a one-click dispute summary the owner can send to the supplier

The dashboard shows today's ledger, expected orders, discrepancies with evidence, and a supplier scorecard that reveals which supplier keeps shorting you. Dockhand is also an MCP server (spec 2025-11-25), so assistants can answer questions like "Did the Sysco delivery come today?", which I demo with a simulated Alexa+ assistant page.

How I built it

• Ring Partner API: device discovery, and live view over WebRTC/WHEP decoded with aiortc. Events come in through HMAC-verified webhooks, an Event History poller, or a manual trigger (I found no way to get webhooks from the Playground). A Postgres-backed queue dedupes replays.
• Smart capture: instead of a fixed timeout, capture stops once the scene has settled, measured by the most-changed patch of a downscaled frame (a small, distant car barely moves the frame average).
• Vision extraction: the top 3 of 10 sampled frames (scored on sharpness and change) go to Amazon Bedrock, which returns validated JSON.
• Reconciliation: matching deliveries to orders is an assignment problem (scipy linear_sum_assignment) over carrier, time window, and box count, with explicit "leave unmatched" options so the solver never sacrifices a good match to force two bad ones.
• Stack: Python, FastAPI, asyncio, PostgreSQL, Next.js + TypeScript, the MCP Python SDK, Docker Compose. 414 Python tests + 33 web tests, CI on every PR.

Results (every number with its sample size)

On synthetic multi-box composites, Sonnet was clearly ahead (27/36 vs 15/36), which motivates the model cascade below.

Eval data: Porch Piracy (CC BY 4.0) and Packages (Public Domain) from Roboflow Universe, MPJ Package Detection (CC BY 4.0) for the synthetic composites, and Ring Playground footage. Selection rules were committed before any model ran. Supplier history in the demo is clearly labeled as simulated.

Challenges

• The Ring Playground behaves differently from the docs: offline devices accepted live-view sessions but sent no video, the Event History filter was ignored, and clips ended unpredictably with no end signal. I recorded each finding as a test fixture and logged 18 friction entries.
• Bedrock access: on a new Free-plan account, an organization policy blocked Bedrock API keys, and some listed models couldn't be invoked. I switched to SigV4 credentials and probed models before trusting them.
• An honest eval: my first synthetic test images had a design flaw that undercounted boxes. I caught it, documented it, and fixed the 16 affected images under a written amendment before re-running. I report those as a second build, not a held-out result.

What I learned

Measuring before optimizing pays off: the latency breakdown showed 25 of the 33 seconds were a fixed capture timeout, not real work. Benchmarking two models on the same data turned a guess into a defensible cost decision. And documenting where an API surprised me was as valuable as the code.

What's next

• Validate on real delivery video (the motion threshold is tuned on Playground clips only)
• Download recorded clips for late events instead of skipping them
• A model cascade: fast Nova first, escalating to Sonnet for multi-box scenes
• A Ring Appstore listing for small businesses

Built With

• aiortc
• amazon
• amazon-web-services
• asyncio
• bedrock
• docker
• fastapi
• mcp
• next.js
• nova
• postgresql
• python
• react
• ring
• typescript
• webrtc