AEGIS

logo

Inspiration

Every "smart safety" product we looked at makes the same architectural bet: stream the camera to the cloud, run the model there, send back a decision.

That bet has a failure mode. When the network drops and on a real industrial site, it drops the system doesn't degrade. It stops. The dashboard keeps showing green. The machine keeps running. Nobody knows the guardian went blind.

We asked one question and let it drive every decision that followed: what does a safety system do when the network goes down?

The answer became our golden rule: nothing in the safety loop depends on the network. We built AEGIS so that you can pull the cable out of the back of the device, in front of the judges, and the worker is still protected.

What it does

AEGIS is a deterministic edge-AI safety interlock. A QNX-powered Raspberry Pi 5 watches a hazardous zone a machine cell, a robot envelope, a taped-off construction area and, entirely on-device, detects two things: a person crossing a defined danger boundary, and missing PPE (no hardhat, no hi-vis vest).

The instant a person enters a restricted zone without the required PPE, the device asserts a GPIO alarm, cuts the machine-enable relay so the machine physically stops, and speaks a spoken warning aloud from locally-cached audio.

Worst-case latency from photon to alarm: 155 milliseconds. Measured, not estimated. On a microkernel, with a bounded worst case not an average. A safety engineer never asks for the average. The cloud is a second brain, not the reflex. MongoDB Atlas is the compliance record. Gemini reviews incidents after the machine has already stopped. Both are architecturally forbidden from making a safety decision.

Reflex versus reasoning. This is the thesis, and it's the thing we most want a judge to take away. The reflex runs on a QNX 8.0 microkernel on a Pi 5, with a bounded 155ms worst case, deterministic, requiring no network and holding no socket and it is the only thing permitted to decide whether the machine stops. The reasoning layer runs on Node, MongoDB Atlas, and Gemini, takes one to three seconds with no bound at all, is stochastic, requires the network, and is never allowed to touch a safety decision.

Most teams put the LLM in the critical path. We deliberately did the opposite. You cannot let a stochastic model behind a variable-latency network call decide whether to stop a machine that can take someone's arm off. So we didn't. Gemini sits behind a hard safety boundary and does what it is genuinely superhuman at: looking at the keyframe, correlating it against a thousand past incidents in MongoDB, and telling a safety manager something they didn't know. It's a reasoning layer, not a reflex. That separation is the architecture.

How we built it

Everything in AEGIS is organised around one line on the architecture diagram: the safety boundary. Inside it sit four QNX processes that cannot fail. Outside it sits everything that is allowed to crash the uplink process, the network, the gateway, MongoDB, Gemini, ElevenLabs, the dashboard. Nothing below that line can affect the alarm. By construction, not by convention.

Five processes. Five memory-protected address spaces. Five priorities. Any one of them can be killed and the system responds correctly.

aegis_capture runs at SCHED_FIFO priority 20 on core 1. It owns the camera via the QNX Sensor Framework and writes NV12 frames into a lock-free shared-memory ring. It does zero AI work. If it dies, the safety controller detects a stale-frame timeout and enters FAIL_SAFE.

aegis_vision runs at priority 15 on cores 2–3. TFLite INT8 inference, the PPE heuristic, and the centroid tracker. It emits a DetectionFrame to safety via QNX MsgSend. If it dies, safety detects a stale-detection timeout and enters FAIL_SAFE.

aegis_safety runs at priority 30 on core 0 the highest in the system. This is the heart. Zone geometry, PPE rules, the state machine, the watchdogs, the GPIO write. No network stack. No heap allocation after init. No third-party libraries. Roughly 400 lines. It doesn't crash that's the entire point of making it this small.

aegis_alarm runs at priority 5 on core 3. It drives the LEDs, buzzer, and relay, and plays the cached voice. It sits deliberately below safety so that audio I/O can never delay the GPIO write.

aegis_uplink runs at priority 10 and is the "allowed to fail" process WAL, HMAC, batching, backoff. It is the only process in the entire system that holds a socket. Kill it and nothing happens to safety. By design.

We verified all of this on target with pidin: capture at 20f in RECEIVE, safety at 30f in RECEIVE, vision at 15f RUNNING, alarm at 5f in REPLY, uplink at 10f in NANOSLEEP. The priorities aren't a claim in a README they're observable on the running device.

Three IPC mechanisms, each chosen for a reason. Capture to vision uses a lock-free shared-memory ring with a seqlock. Four slots at /aegis_frames. The writer bumps a sequence counter to odd, writes, bumps to even. The reader reads the sequence, reads the data, re-reads the sequence if it changed or is odd, it retries. The consequence is that the reader can never see a torn frame and the writer never blocks on a slow reader. Frames are dropped, never delayed, because for a real-time safety system a fresh frame beats a complete-but-stale one. We used explicit memory_order_release on publish and memory_order_acquire on consume rather than seq_cst everywhere we reasoned about each one.

Vision to safety uses native QNX Send/Receive/Reply. The sender blocks until we reply, which gives us flow control and backpressure for free: a runaway vision process cannot flood the safety controller, because the rendezvous physically prevents it. This synchronous, blocking IPC model is a defining Neutrino microkernel property, and we rely on it as a correctness guarantee rather than a convenience. We handle _PULSE_CODE_DISCONNECT so that when a client dies, safety is notified immediately rather than hanging.

One detail we're proud of: run pidin and you'll sometimes see the safety thread reporting priority 15 while blocked in RECEIVE. That's priority inheritance the receiving thread transiently adopts the sender's priority to prevent priority inversion, and snaps back to 30 the instant it has work. That's not a bug we tolerated. That's the microkernel doing exactly what a hard-real-time kernel is supposed to do.

Safety to uplink uses a lock-free SPSC queue, fire-and-forget. If the queue is full, safety drops the event and increments a counter. It never blocks. The comment in the source is our thesis statement: the safety loop must never, under any circumstances, be slowed down by the telemetry path. If the cloud is slow, the cloud loses data. The worker does not lose a hand.

The safety state machine is explicit and exhaustive an enum and a transition function, not a tangle of ifs. Four states. SAFE is green and quiet. WARNING fires when a person comes within a 15% margin of the zone boundary we warn before the violation. VIOLATION means a person is inside a zone and failing its PPE requirement: red LED, buzzer, relay cuts, machine stops. FAIL_SAFE means a watchdog expired: red at 4Hz, relay low, and the device announces "system fault, assume unsafe, machine stopped."

The distinction between the last two is the one we most want a judge to hear. VIOLATION means "I can see a hazard." FAIL_SAFE means "I have lost the ability to see, therefore I must assume the worst." A safety system that goes quiet when its sensor dies is not a safety system.

FAIL_SAFE is latching. It does not clear on one good frame. It requires five seconds of continuous healthy operation or an explicit operator reset. Hysteresis is five consecutive frames to enter a violation, ten to clear a person jittering on a zone boundary must not produce alarm chatter, because an alarm that cries wolf gets taped over on a real site, and then it protects nobody.

The watchdogs are the insight that makes the whole demo work. Both are driven by a QNX timer (timer_create with SIGEV_PULSE delivered to our own channel) firing every 50ms, so the safety loop wakes up regardless of whether anything else in the system is alive. A watchdog that only runs when its input arrives is not a watchdog. No new frame in 500ms, or no DetectionFrame in 500ms, and we're in FAIL_SAFE.

The GPIO is wired normally-closed. The SAFE state requires an active signal from the Pi. If the Pi loses power, if the safety process dies, if a wire falls out the relay de-energises and the machine stops. The system fails to the safe state, not the dangerous one. That's how real industrial interlocks are built (IEC 61508 / ISO 13849 territory), and getting it backwards means a dead Pi is a silent, dangerous machine.

The on-device AI satisfies the QNX challenge's oss.qnx.com requirement: TensorFlow Lite, cross-compiled for QNX aarch64le from the qnx-ports build system, running on the embedded hardware and not in the cloud. SSD-MobileNet, 300×300, INT8 quantized, with the XNNPACK delegate confirmed loaded on target. Mean inference is 51ms measured on the Pi, not on a laptop sustaining 10–13 FPS. All tensors are allocated once at startup; the hot loop never allocates, because heap allocation in a real-time path introduces unbounded latency and is a cardinal sin in embedded safety code. We deliberately set SetNumThreads(3), not 4, leaving a core for the safety process: throughput is not the goal, bounded predictable latency is. CPU core isolation is explicit via ThreadCtl(_NTO_TCTL_RUNMASK) safety owns core 0, capture owns core 1, vision gets 2 and 3 so the real-time path is isolated from general OS scheduling.

The PPE call is a heuristic on purpose, and it's explainable. The detector gives us the person box; a second-stage HSV pass gives us the PPE verdict. For a hardhat we analyse the top 22% of the bounding box, convert to HSV, and measure the fraction of pixels falling in configurable hardhat-colour ranges — yellow, white, orange, blue, red. For a vest we do the same on the torso region, 25% to 65% of box height, looking for fluorescent yellow-green and orange. Every threshold is hot-reloadable from /etc/aegis/aegis.json, because we knew we'd be re-tuning under venue lighting at 2am, and hardcoded thresholds cost you the demo. We emit confidence values, not booleans, so the dashboard can show exactly why we flagged something. Showing our reasoning beats asking anyone to trust a black box.

A centroid tracker gives each person a stable track_id across frames. Without it the dashboard flickers and violations re-fire for the same person. With it, we can say "Person #3 has been in violation for 4.2 seconds," which is far more actionable than "a violation occurred."

We refused to let a third-party library be a single point of failure. All OpenCV usage sits behind an IImageOps interface with a dependency-free fallback that implements bilinear resize, RGB→HSV, and ROI extraction by hand. One CMake flag (AEGIS_USE_OPENCV=OFF) and we keep shipping. We didn't need it but the option existed from hour one, and that's the point.

The offline story is where we put our money. Every event is appended to a write-ahead log on disk at /data/aegis/buffer/events.wal with a length prefix and a CRC32 per record, before any network attempt, with periodic fsync() so a power cut loses at most one batch. A durable read cursor tracks the byte offset of the last event the cloud acknowledged with a 2xx and the cursor only advances on an ACK. Never on a send. Never on a hope. On startup we resume from the cursor, so a reboot, a crash, or a kill -9 loses nothing.

The transport uses exponential backoff with ±20% jitter (1s, 2s, 4s, capped at 30s), where the jitter exists to prevent a thundering herd when a fleet of devices reconnects simultaneously. Timeouts are aggressive 2s connect, 5s request because a safety device must never have a thread hung on a TCP connect to a dead server. Every event carries an idempotency key, its event_id UUID, which the cloud upserts on: at-least-once delivery plus idempotent writes equals effectively-once, which is the correct guarantee for telemetry. Every batch is signed HMAC-SHA256 and verified server-side with a timing-safe comparison, because a safety device's telemetry must be authenticated an attacker who can forge SAFE events into the historical record can hide an incident.

Every event carries a buffered_offline flag and a buffered_duration_ms, and the dashboard renders buffered events in amber, so when the backlog flushes a judge sees the wave pour in.

The measured result, queried directly from Atlas rather than trusted from a log line: 7,290 total events in the database, 1,200 from the real Pi, of which 638 were captured while the device was disconnected. Events lost: zero. A cloud-dependent system would have recorded none of those 638 and protected nobody for the entire outage.

MongoDB Atlas is not a key-value store here — every feature was chosen because it was the right tool for a specific job. telemetry is a genuine time-series collection (verified "type": "timeseries" via listCollections, timeField ts, metaField meta, granularity seconds) because columnar storage compresses a 1–15Hz sensor stream dramatically better and optimises time-bucketed aggregation. It carries a 7-day TTL, because high-frequency telemetry is disposable and incidents are not — different retention for different data classes is a data-modelling decision, not an afterthought. The events collection has a unique index on event_id, which is precisely what makes the uplink's at-least-once retries idempotent, and is the correctness argument for the entire ingest path. jsonSchema‘validationguardstheboundary,becausethedatabaseisthelastlineofdefencefordataintegrity:amalformedeventfromabuggydevicemustberejectedatthedoor,notdiscoveredthreedayslaterinacomplianceaudit.KeyframesliveinGridFS(58+stored)tokeep‘events‘smallandfasttoquery.Changestreamspushtothedashboardwithzeropolling,withresumetokenspersistedsoagatewayrestartmissesnothing.AtlasVectorSearchrunsoverthe‘events.embedding‘field—indexREADYandqueryable,768dimensions,cosinesimilarity,withfilterson‘deviceid‘,‘eventtype‘,and‘safetystate‘—becauseasafetymanagerdoesn′twanttogreplogs,theywanttoask∗hasthishappenedbefore∗andgetthethreemostsimilarpastincidents.Andwecomputep50/p95/p99latencywithserver−side‘jsonSchema validation guards the boundary, because the database is the last line of defence for data integrity: a malformed event from a buggy device must be rejected at the door, not discovered three days later in a compliance audit. Keyframes live in GridFS (58+ stored) to keep events small and fast to query. Change streams push to the dashboard with zero polling, with resume tokens persisted so a gateway restart misses nothing. Atlas Vector Search runs over the events.embedding field — index READY and queryable, 768 dimensions, cosine similarity, with filters on device_id, event_type, and safety_state — because a safety manager doesn't want to grep logs, they want to ask has this happened before and get the three most similar past incidents. And we compute p50/p95/p99 latency with server-side jsonSchema‘validationguardstheboundary,becausethedatabaseisthelastlineofdefencefordataintegrity:amalformedeventfromabuggydevicemustberejectedatthedoor,notdiscoveredthreedayslaterinacomplianceaudit.KeyframesliveinGridFS(58+stored)tokeep‘events‘smallandfasttoquery.Changestreamspushtothedashboardwithzeropolling,withresumetokenspersistedsoagatewayrestartmissesnothing.AtlasVectorSearchrunsoverthe‘events.embedding‘field—indexREADYandqueryable,768dimensions,cosinesimilarity,withfilterson‘deviceid‘,‘eventtype‘,and‘safetystate‘—becauseasafetymanagerdoesn′twanttogreplogs,theywanttoask∗hasthishappenedbefore∗andgetthethreemostsimilarpastincidents.Andwecomputep50/p95/p99latencywithserver−side‘percentile rather than pulling raw documents into Node, which is what matters when a fleet has ten thousand devices.

Our aggregation pipelines cover violationsByHour, ppeComplianceRate, latencyPercentiles, deviceHealth, and the one we care about most offlineResilienceReport, which counts events where buffered_offline is true, bucketed by outage duration. That pipeline literally quantifies how many safety events a cloud-dependent system would have lost. It is our thesis rendered as a data visualisation.

Gemini is an agent, not a chatbot. Violation keyframes from GridFS plus structured event context go to Gemini with a strict responseSchema and responseMimeType: application/json it cannot return prose, it must return valid parseable JSON matching our schema.

The feature we're proudest of is cross-validation. We explicitly ask Gemini whether it agrees with the on-device detector. The edge said "no hardhat," in 51ms, deterministically does the image support that? The dashboard renders both verdicts side by side: the edge detector's call and confidence on the left, Gemini's verdict and reasoning on the right, and between them a CONFIRMED or DISPUTED badge. A fast deterministic detector made the safety decision in 51 milliseconds. A slower reasoning model reviewed it afterwards. The machine was already stopped before Gemini was even called. A disagreement is surfaced as a possible false positive for human review. It's a genuinely useful pattern and we haven't seen another team do it.

Gemini has real tools that reach into MongoDB: query_incidents(window, zone_id, violation_type) runs an aggregation, get_compliance_rate(window) runs ppeComplianceRate, find_similar_incidents(event_id, k) runs Atlas Vector Search, and get_device_health(device_id) returns device status. The full multi-turn loop runs question plus tool declarations, functionCall returned, we execute against Mongo, functionResponse sent back, Gemini reasons over real data, final answer. Verified live: the agent called query_incidents(window=last_shift) and got back 7,209 rows from real Mongo data. A safety manager can type "has this kind of violation happened before near the press?" and get an answer backed by an actual database query.

And we show the tool calls. The UI renders an inline card as Gemini invokes each function, naming the call and the row count returned. We refuse to hide the agent's actions; watching it reach into the database and come back with real numbers is the wow-factor, and hiding it makes an agent look like a chatbot. The UI says so explicitly: agent actions are shown, not hidden.

The embeddings are where Gemini and MongoDB genuinely need each other. gemini-embedding-001 with outputDimensionality: 768 embeds a textual summary of each incident; the vector is written back to the event document and indexed by Atlas Vector Search. Gemini generates the semantic representation, Atlas indexes and retrieves it, and Gemini's function-calling agent queries it back. Neither integration is bolted on they are load-bearing for each other.

We designed for our own AI to fail. Every Gemini call has a hard 10s timeout and a retry with backoff, wrapped in a circuit breaker that opens after N failures and serves an honest degraded response. On a 429 we walk a model fallback chain. The dashboard displays a visible AI LAYER: DEGRADED badge. We tested it with an invalid API key: the system remains 100% functional, events still write to Mongo with gemini_analysis: null and a structured skip_reason. Demonstrating that you designed for your AI to fail is far more impressive than pretending it won't.

ElevenLabs is baked in, not fetched. There are two voice layers and the split between them is the whole point. Online, the dashboard streams expressive narration of Gemini's incident analysis, cached in GridFS keyed by a hash of the text so we never pay to synthesise the same sentence twice. Offline the part that matters a build-time script pre-generates six critical alarm phrases as WAVs and ships them to the Pi: "Warning. You are approaching a restricted zone." / "Stop. Hard hat required in this area." / "Stop. High visibility vest required." / "Danger. Restricted zone. Exit immediately." / "System fault. Assume unsafe. Machine stopped." / "Area clear. Safe to proceed." The voice is "Adam — Dominant, Firm," chosen deliberately for a clear, urgent, authoritative safety-announcer tone. Not a cheerful assistant. Voice design is a real decision and it signals intent.

These play from local disk with zero network access. The voice a worker hears in an emergency was generated by ElevenLabs. It is not fetched from ElevenLabs. A safety alarm that requires an API call to speak is not a safety alarm. And the ordering is itself a real-time design decision: audio playback runs in aegis_alarm at priority 5, below the safety process at 30. The LED and the relay fire first. The voice follows. Audio I/O can never delay the GPIO write.

The dashboard is a control room, not a SaaS panel. Next.js 14 App Router, TypeScript strict, Tailwind, framer-motion, recharts. The design language is SCADA air traffic control, Bloomberg terminal. Near-black #0A0B0D. Monospaced tabular numerals so digits don't jitter as they update. Colour is purely semantic: cold industrial green for safe, amber for warning, saturated red for violation, and a visually distinct magenta for FAIL_SAFE, because "I can see a hazard" and "I have gone blind" must never look the same. No decorative colour, ever. The state of the system must be legible from three metres away.

The live console carries a full-width state banner, the camera feed with zone polygons and bounding boxes and PPE badges and track IDs rendered on canvas, a per-link status panel, and a capture-to-alarm latency sparkline with the 200ms budget line drawn across it. The offline banner, DEVICE OFFLINE — SAFETY SYSTEM STILL OPERATIONAL ON-DEVICE, with a live buffered-event counter is the single most important UI element in the entire project. Incident detail pages show the keyframe with overlays, the edge-versus-Gemini cross-validation panel, the ElevenLabs narration player, and a "similar past incidents" rail powered by Atlas Vector Search. The agent console streams token-by-token with visible tool calls. Analytics shows PPE compliance over time, a violations-by-zone heatmap, latency percentiles captioned as computed server-side via $percentile, and the offline resilience panel. And a device health view renders the five processes with their priorities and CPU affinities like a control-room process table a general judge who has never heard the word microkernel will still see that something serious is happening.

Accessibility: aria-live regions announce state changes to a screen reader, and no state is signalled by colour alone every state carries an icon and a text label.

Challenges we ran into

The QNX camera path is officially experimental, and we timeboxed it rather than gambling on it. The QNX Sensor Framework CSI path on the Pi 5 is not a validated pipeline, and QNX boots with a colour-bar simulator bound to the sensor service by default. Many teams would have spent ten hours discovering that their "working camera" was a test pattern. We gave it a hard three-hour timebox and built ICaptureSource an interface with a QSF implementation and a file-replay implementation, selectable with --source=qsf|file from day one, before we knew whether we'd need it. We got the real camera working: live IMX708 frames, NV12, negotiated at 1536×864. But the fallback existed from hour one, and that's the point. We also wrote a frame_dump tool whose sole purpose was to prove we weren't looking at the simulator: it prints mean, standard deviation, min, and max pixel statistics. Wave your hand in front of the lens; if the numbers don't move, you're looking at a test pattern. That tool caught the number-one false positive in this entire challenge.

The zero-loss flush bug that would have silently destroyed our headline claim. Our uplink buffered 51 events during an outage, POSTed all 51 in a single batch, the gateway rejected it with 413 batch too large (max 50), and the uplink logged "flush complete" and dropped all 51 events on the floor. Our entire "zero data loss" claim was false, and every log line said it was fine. We fixed it with chunked flush and ACK-only cursor advance: the cursor now advances only on a 2xx from the gateway, never on a send. The lesson we'd put on a slide is that a resilience claim you haven't tried to break is a marketing claim. We only found this because we went looking for it with a hammer.

Stale shared-memory mappings a real distributed-systems bug hiding in a single box. Restarting aegis_safety alone calls shm_unlink() and creates a fresh /aegis_events segment, but aegis_uplink keeps its existing mmap, now pointing at orphaned physical pages. The result: safety logs "queue full, dropped" while uplink logs "drained 0 events." Both processes healthy. Both lying. Zero events reaching the cloud. Two processes, two mappings, one name, different memory. Our operational fix is that you never restart the safety process alone; the proper fix refcounted segments with generation numbers is on the roadmap, and we know exactly what it looks like.

The Pi 5 has no battery-backed RTC, so we stopped trusting the device clock. Every event arrived timestamped 1970-01-01, because the C++ uplink stamps with CLOCK_MONOTONIC, which starts at zero on boot. Events sorted to the bottom of the collection. Analytics went blank. The dashboard permanently read DEVICE OFFLINE, because it computed liveness as now minus event timestamp and every event looked 56 years stale. Gemini's agent queried "last shift" and got nothing back. We fixed it at the gateway rather than on the device, and that was deliberate: a device without an RTC cannot be trusted to tell you what time it is. The gateway stamps ingest time and preserves the device's monotonic value separately, for latency math, which is the only thing a monotonic clock is actually valid for. That's how you handle clock skew across a real fleet.

The HMAC raw-body bug. Fastify's default JSON parser re-serialised the body before HMAC verification, so signatures from the C++ uplink never matched. Fixed by removing the default content-type parser and installing a raw-string parser. The C++ uplink signs raw bytes, so the gateway must verify raw bytes.

text-embedding-004 is retired; it isn't in the live /models list. We migrated to gemini-embedding-001, which is natively 3072-dimensional. Without explicitly passing outputDimensionality: 768, Atlas Vector Search silently returned nothing at all. No error. Just empty results.

Full-frame bounding boxes broke the zone geometry. Standing too close to the camera, the detector returned a bounding box spanning 92% of frame width and 100% of frame height. Our zone engine uses the bottom-centre of the bbox as the ground-contact proxy; a person's centroid can be over a zone while they stand outside it, so feet, not centre, is the correct model. With no feet in frame, the geometry was meaningless: the centroid jumped every frame, the tracker spawned a new track ID every two frames, and every new ID reset the hysteresis counter so the state strobed between VIOLATION and WARNING several times a second while we stood perfectly still. We fixed it by loosening the tracker's match distance, extending track max-age, hardening hysteresis to five-in and ten-out, and, critically, fixing the camera framing. The lesson: your hysteresis is worthless if your tracker can't hold an identity.

Accomplishments that we're proud of

We pulled the Ethernet cable out of a running device, and the alarm kept firing. Everything else is commentary.

638 events were buffered offline, and zero were lost, verified by querying Atlas directly, not by trusting a log line.

155ms worst-case photon-to-alarm. Not a mean. A worst case, on a microkernel, with a bounded guarantee. A safety engineer never asks for the average.

51ms on-device INT8 TFLite inference with XNNPACK real AI, on real embedded hardware, not behind a cloud API.

Five memory-protected processes at five priorities, verified with pidin on the target. Killing the AI process triggers FAIL_SAFE in under 500ms. Killing the network process does nothing to safety. Both demonstrated live, on demand.

42 unit tests including a 10,000-case fuzz test on the safety state machine hammering it with NaN coordinates, out-of-range person counts, and negative confidences. It never crashes, never reads out of bounds, and always ends in a valid state. Safety code must be robust to garbage from a buggy upstream process, and we proved ours is.

A safety process small enough to read end-to-end in ten minutes. When a judge asks how we know it works, the answer is: because it's small enough to prove.

And we designed for every one of our dependencies to fail then killed them one by one to check.

What we learned

A resilience claim you haven't tried to break is a marketing claim. Our zero-loss flush was broken and every log line said it was fine. We only found it because we went looking with a hammer.

The hardest part of real-time isn't speed; it's bounded speed. A 30Hz system with unbounded jitter is worse than a 5Hz system with a rigorously defended 200ms worst case, and any RTOS engineer knows it.

Priority inheritance is real, and it looks like a bug until you understand it. Seeing a priority-30 thread report 15 sent us hunting for an hour. It was the microkernel preventing priority inversion, working exactly as designed.

Shared memory across process restarts is a lifetime-management problem, not a memory problem. Two processes, one name, different physical pages, both convinced they were healthy.

Designing for your own AI to fail is a feature, not a hedge. The circuit breaker is one of the things we're most proud of, because building it forced us to answer a question: what is this system without Gemini? The answer a fully functional safety interlock, which is the whole thesis.

What's next for AEGIS

A functional-safety certification path. The architecture was built with IEC 61508 and ISO 13849 in mind: a small auditable safety kernel, fail-safe normally-closed actuation, a latching fault state, watchdog-driven fault detection, and a rigorous separation between safety-rated and non-safety-rated code. Getting SIL-rated is a process, and we designed to make that process possible rather than impossible.

A properly trained PPE model. Our HSV heuristic is explainable and fast, but it is a heuristic. A labelled worksite dataset and a purpose-trained INT8 detector replaces it while keeping the explainability layer, because a safety manager needs to know why.

Refcounted shared-memory segments with generation numbers, so a process restart can never orphan a peer's mapping.

Signed OTA model updates with rollback. You cannot push an unverified model to a device that stops machines.

Multi-camera sensor fusion. One camera has occlusion. Two have parallax and a far better ground-plane estimate.

Fleet management for thousands of devices the backoff-with-jitter and idempotent-upsert designs were chosen with this in mind from day one.

A hardware watchdog IC. If the safety process itself hangs, an external watchdog must reset the board. We note in the source exactly where it belongs.

The numbers, for the record Worst-case photon-to-alarm: 155ms, against a 200ms budget the headline number, and a worst case rather than an average. On-device TFLite inference: 51ms, XNNPACK-accelerated. Safety process: SCHED_FIFO priority 30, core 0, verified with pidin. Watchdog detection from sensor loss to FAIL_SAFE: under 500ms. Events buffered during network outage: 638. Events lost: 0. Safety unit tests passing: 42, including a 10,000-case fuzz test. Lines of code in the safety-critical process: ~400 small is the point, because small is auditable. QNX processes and address spaces: five and five fault isolation is architectural, not aspirational.