-
-
demo takes ~30sec to load
Inspiration
Two nights before the hackathon I tripped over a charging cable in my own
apartment. I was walking back from the kitchen at 3 a.m. with the lights
off, on autopilot, and I went down hard enough to bruise a hip and split a lip. It was the third time this year. Not a great look for someone whose
entire degree is about teaching machines to perceive space.
Lying on the floor doing that staring-at-the-ceiling reassessment thing, I realized something embarrassing about the apartment around me. My smart
speaker knows my voice. My phone knows my schedule. The robot vacuum has a millimeter-accurate map of every wall. Not one of those systems was, at
any point that night, checking that I had not left a tripwire across my
own home — even though every one of them had the sensor data to do it.
It scaled in my head. The same gap exists, with much higher stakes,
everywhere embodied AI is shipping right now: humanoid-class policies
moving into homes (Figure, 1X), warehouse AMRs operating around human
pickers, Autopilot eating pedestrians on a quarterly cadence. We have
built a stunning content-moderation stack for AI's outputs and
approximately nothing for AI's actions. There is no
"is-this-output-safe" classifier equivalent for "did the robot just enter
the nursery."
So I built one. Or, the layer one would call into.
### What it does
Penumbra is a real-time, browser-only reference implementation of a
spatial constitution enforcer for embodied AI. The operator declares
geometric constraints — the robot does not enter the nursery, the drone
stays out of the cyclist lane, no machine passes within 1.5 m of a tracked human — and Penumbra continuously verifies them against a live 360°
LiDAR feed. Every decision is logged into a tamper-evident hash chain.
Open the live site, pick a scenario, watch the synthesized 360° point
cloud sweep around the agent, paint a no-go zone with your mouse, watch
the simulated agent walk into it, see the violation flash and the audit
chain advance, toggle "inject spoof points" and watch the sensor-integrity meter blow past suspect threshold while the chain still records every
event.
### How I built it
Synthesized point cloud from a Velodyne-class sensor model → voxel-grid
acceleration → DBSCAN clustering → multi-object Kalman tracking with
greedy gated assignment → per-zone signed-distance-field evaluation
against the agent pose and every confirmed track → decisions appended to a hash-chained audit log → all of it visualized in Three.js with a custom
intensity-shaded point shader. Twenty-Hz perception, 60 Hz render. About
1.5k lines of perception/constraint code, plus an audit log and the UI.
The whole stack is browser-only — no backend, no build step, just HTML +
ES modules + Three.js loaded from a CDN via importmap. That choice was
deliberate: judges should be able to open one URL and see it work, and the source they read is literally the source that runs.
### Challenges I ran into
The point shader. I burned an hour on a shader where points were
correctly positioned but invisible. It turned out to be the depth write —
transparent points need depthWrite: false so a near-camera sprite
doesn't occlude points behind it.
DBSCAN's neighborhood query in JS. Naive forEach over points hit ~28 ms per sweep at 12k points and the demo started thrashing. Voxel-grid
acceleration brought it under 4 ms.
The honest version of the audit chain. I started with a synchronous
SHA-256 fold, then realized WebCrypto is async-only. Switched to FNV-1a
for the per-tick chain (fast, sync, not collision-resistant) and exposed sha256Hex for the exported chain. The README is explicit about this
trade-off — silent cryptographic substitutions are how you ship broken
safety code.
### Accomplishments I'm proud of
A real, runnable, end-to-end embodied-alignment demo in a browser tab.
~1.5k lines of perception code, no stubs. A defensible technical thesis
(docs/ALIGNMENT.md) that doesn't hand-wave the alignment angle. A spoof
detector calibrated to a real attack class (Cao et al. 2019). And a
hash-chained audit log that's actually verifiable — flip a byte in any
past decision and every subsequent hash breaks.
### What I learned
That the gap between "AI alignment for text" and "AI alignment for
actions" isn't a research problem yet — it's an infrastructure problem.
The text-side has constitutional AI, RLHF, content classifiers, and
red-teaming pipelines that compose. The embodiment side has none of the
equivalents. It's not because the math is harder; it's because nobody has
built the verifier layer.
### What's next for Penumbra
In rough order of how directly they unlock real deployments: real LiDAR
ingestion (RPLidar A3 over WebSerial, ~one evening), planner integration
contracts, TEE-anchored audit, cross-modal verifier (camera + LiDAR), and
a community-maintained zone library — the "constitutional library"
equivalent for spatial constraints.
Log in or sign up for Devpost to join the conversation.