Rangyuu

Inspiration

A wheelchair, a mobility scooter or a small robot can see what is in front of it. It cannot see around the corner. But the building it is moving through often already has cameras looking into that exact space. We wanted to know whether those cameras could answer one narrow, honest question for anything that moves: can I traverse this path right now? — and, just as important, say "I don't know" when they can't.

What it does

Rangyuu turns Ring camera observations into a floor-plane occupancy map and answers a route query with one of three words:

Formally, for a path $P$ made of floor cells:

$$ \texttt{PROCEED} \iff \forall c \in P:\ \text{free}(c) \land \text{fresh}(c) \land \text{calibrated}(c) \land \text{conf}(c) \ge 0.5 $$

Anything else withdraws permission. Absence of evidence is never a yes: cut the feed and the answer drops from PROCEED to UNKNOWN without anything else changing.

The app has three screens — Now (decision, reason, the map cells behind it), On board (the vehicle's own phone camera) and System (sources, measured calibration error, language) — in English and Spanish. A thin Expo client speaks state changes with pre-recorded voice clips, so the "link lost" alert works offline.

Rangyuu is meant for supervised use: a parent, caregiver or operator stays responsible. It never steers or brakes anything.

How we built it

• Ring as the infrastructure sensor. A Python client calls the Ring API (devices, status, capabilities, location), opens a WHEP live-view session without a browser, and decodes the H264 media with aiortc. Ring webhooks are verified with HMAC-SHA256 over the raw body, in constant time, before any JSON is parsed.
• A floor map, not a picture. A normalized DLT homography $H$ projects each detection's ground point onto the floor plane, $\tilde{x}{floor} \sim H\,\tilde{x}{image}$. Each cell carries a state, a confidence and an age, and it expires.
• Calibration has to be measured. Error is reported on points left out of the fit, not the fit residual. An uncalibrated or merely fitted camera can observe but never authorize.
• Local and fast. Decode, inference and decision run on the machine; no third-party inference is called.
• Tested end to end. pytest plus Playwright browser tests: 206 passed, 1 skipped.

Measured, with dates and repeatable commands in the repo:

• Full loop on a live WHEP session from the Ring Developer Playground: 400 frames at 1280×720, 8.16 ms per frame (decode 7.48, inference 0.58, decision 0.10).
• Floor calibration: 0.9 cm error over 61 cm (1.6 %), on a tape-measured floor filmed with a phone.
• On-board camera lane: 22 % precision against a labeled walk — so it is only allowed to warn, never to stop anyone.

Challenges we ran into

• The Playground live view is a canned ~40 s clip (1172 frames, same in every session, no floor geometry). It exercises the integration but you can't calibrate against it, so we measured calibration on our own tape-measured footage and say so.
• Short-lived Playground tokens (~30 min) during long measurement runs.
• Our own on-board detector wasn't good enough. At 22 % precision we demoted it from "can stop" to "can only warn" instead of hiding the number.
• A single-threaded server froze when a browser held an idle connection; we made it threaded and added a regression test.
• WHEP outside a browser: getting SDP offer/answer, session Location and DELETE teardown right from Python.

Accomplishments that we're proud of

• A safety contract where only one word authorizes and silence, staleness and missing calibration all fail closed.
• Ring live view consumed from a server process, decoded and decided in ~8 ms per frame.
• Every number in the README has a date and a command to reproduce it, and a "Not yet verified" list states what we haven't proven.
• An open-source contribution back to Ring's own hello-world sample: a terminal WHEP live-view script.

What we learned

• "Unknown" and "clear" are different answers, and the system has to be designed so the first never turns into the second by accident.
• Fit residuals flatter you; held-out error tells the truth.
• A weak sensor can still be useful if its authority matches its measured precision.
• WHEP is plain HTTP signaling, which makes Ring cameras usable as infrastructure sensors, not just doorbells.

What's next for Rangyuu

• Feed the app directly from a live Ring WHEP session (today the demo app runs on a simulated floor feed).
• A measured, per-device calibration for the webhook path, and calibration with a physical Ring camera beyond 1.5 m.
• OAuth app linking so real Ring webhooks reach Rangyuu.
• Glass-to-glass latency on physical hardware.
• A small on-vehicle unit that connects to the Ring devices a building already has (planned, not built).

Built With

• aiortc
• css3
• elevenlabs
• expo.io
• hmac
• html5
• javascript
• numpy
• onnx-runtime
• playwright
• pytest
• python
• react-native
• ring-api
• ring-developer-playground
• typescript
• webrtc
• whep