AFTERIMAGE

Every monitor detects what happened. AFTERIMAGE detects what didn’t: the caregiver who never came, the dose not taken, learned from your own Ring history, and honest when it can’t tell.

Comment

Inspiration

Every monitoring product ever built is organized around presence. A camera sees motion, a sensor closes a circuit, an alert fires. Ring records 257 events in Eleanor’s house over three weeks, and on the day everything goes wrong, not one of those events is the thing that went wrong.

What actually fails in a home where somebody is being cared for is almost never an event. It is a non-event. The caregiver who did not arrive. The pills that were not taken. The walk that was not walked. These produce no footage, trip no sensor, and generate no alert, which is precisely why they are the failures that get missed. A daughter forty minutes away finds out at nine in the evening, if she finds out at all.

We wanted to build the thing that notices nothing happening, and to find out whether an absence can be reported with enough evidence that a family would actually leave it switched on.

What it does

AFTERIMAGE learns what a household normally does from its own Ring history, then checks every day against that and reports the silences that matter.

It takes 21 days of Ring events and learns 11 routines with no schedule ever entered: the morning dose at 9:08 AM plus or minus 27 minutes on 100 percent of applicable days, Marcus the caregiver at 8:44 AM, his car at 8:41, the garden door at 7:20 AM, the deadbolt at 9:39 PM. Each routine carries its own timing spread, its support, and a confidence.

Then comes Tuesday. The patio door opens at 3:12 AM, which in 21 days has never happened at night. The garden door goes at 7:58 AM, 38 minutes later than it has ever gone. Marcus’s car and then Marcus never arrive. At 9:08 AM the medicine cabinet does not open, breaking a run of 21 days out of 21, and that finding scores severity 91. The mail arrives at 2:34 PM exactly as normal and the system says nothing, on purpose. A face the house has never matched appears for 22 seconds at 4:10 PM. An unfamiliar vehicle sits in the driveway for 41 minutes at 6:55 PM, rarity 0.90. At 9:39 PM the deadbolt is still pending, because the window has not closed, so nothing is wrong yet.

Four things make those claims defensible:

It refuses to learn weak patterns. Any pattern whose timing spread exceeds 30 minutes is thrown away, on the grounds that an event you can only place inside a two hour window is not something a household can act on. The neighbour’s cat on the patio never becomes an expectation. It separates silence from blindness. The doorbell was on its charger from 3:12 PM to 4:04 PM, so it saw only 25 percent of the window around the 3:36 PM walk. The verdict is “cannot tell” with confidence 0.09, not “did not happen”. Unknown is a first class verdict in the type system and it propagates all the way through to the spoken answer. It claims a cause only where the statistics allow one. The morning dose follows Marcus’s arrival within 24 minutes, varying by only plus or minus 2.8 minutes across 13 of 13 days where both applied, against a plus or minus 13 minute spread when measured against the clock. That coupling scores 0.97 and produces the sentence “No care visit today. Most likely why there was no morning dose.” The garden door happens every morning too, fails the same test, and produces no link at all. It ranks by consequence, not by confidence. The engine works out what recurs and cannot know what a missed dose costs, so the weights come from the household’s care plan. A near certain missed delivery (severity 0) sits below a less certain missed dose (severity 91).

It then closes the loop. Eleanor is asked on her own Echo at severity 55, Dana at 70, the care agency at 80, and Ruth the neighbour at 92, which today’s worst finding does not reach, so Ruth is not called. Each contact waits out the previous one’s window, 45 minutes then 60 then 90, so the household is escalated through rather than all woken at once, and every queued notification shows the exact words that would be said. Say “I was with her, it is fine” and that finding greys out and stops escalating.

The same WorldState drives three Amazon surfaces, so the spoken sentence and the evidence card under it cannot disagree. Ring is observation: 8 devices across 6 zones, an event stream, coverage gaps, and a simulator that pushes events, moves the clock, and blinds cameras. Fire TV is reconstruction: the day as a dial, findings ranked, evidence on Select, D-pad only. Alexa+ is reasoning: each utterance resolves to one tool call such as afterimage.explain_finding or afterimage.list_coverage_gaps, with the call and its structured result shown in the rail as the answer is spoken, including the answer to “what can you not tell me?”

How we built it

TypeScript and React 18 on Vite 5, with no backend and no database. The whole thing is a pure function of an observation set.

The engine lives in src/engine and has no UI imports at all, so it runs under plain Node: types.ts makes observed, expected and unknown a type rather than a convention; scenario.ts holds the devices, the care plan and a deterministic seeded generator for 21 days of Ring history; routines.ts learns recurring expectations and rejects formless ones; absence.ts produces the verdicts satisfied, late, pending, uncertain and missing; novelty.ts scores events with no precedent against this house only; correlate.ts computes the coupling statistic that justifies a causal link; reason.ts answers questions over the world state and returns structure rather than strings, specifically so the same function can back both the Alexa+ conversation and a Streamable HTTP MCP server; escalate.ts decides who is told and when; selftest.ts asserts all of it.

The history generator produces genuinely dependent events. The dose is timed off Marcus’s actual arrival rather than off the clock, which is why the statistics the engine discovers are real statistics rather than hard coded conclusions. Take the caregiver away and the dependency shows up in the data on its own.

The UI is four surfaces over shared state: Console, FireTV, AlexaPlus and RingSim, with a day dial that can be dragged and played, a floor plan that answers the same day spatially, an evidence drawer, a 21 day history strip, a transport, and a live feed that announces each verdict as it changes. View state lives in the URL hash, so you can send somebody a link to the medicine cabinet at 9:08 on the morning it was missed. Preferences that belong to a person rather than to a link, the theme and the care weightings, live in localStorage instead.

The submission video is itself code. demo/film.mjs drives a real Chrome against the dev server with playwright-core, clicks the real controls, and draws its own title cards and lower third into the page, so the whole film comes out of one pass with no editing and no imported media. A synthetic pointer and keypress chips are drawn before every click, because an interface that changes with no visible cause reads as a video edit rather than as a product working.

Challenges we ran into

The hard part was never detection. It was earning the right to make a claim.

The off schedule bug. Expectation windows are two standard deviations wide, which means about five percent of perfectly normal occurrences fall outside them. An event eleven minutes early was being reported as “did not happen”. An absence detector that cries wolf at ordinary drift is worse than no absence detector, so that case is now an assertion in the suite.

Causality that is not just correlation. It would be easy and wrong to say the missed dose was caused by the missed care visit simply because both happen daily and one follows the other. Any two daily routines look causal under that test. Measuring the standard deviation of the gap between the two events across history is what separates the dose, at plus or minus 2.8 minutes, from the garden door, which is equally daily and produces nothing.

Coverage as a first class concept. Making “I cannot tell” a real verdict rather than an error case meant threading it through the learner, the absence logic, the ranking, all four surfaces and the voice answers. It would have been far easier to report the afternoon walk as missing and never mention the charging window.

An acknowledgement bug that only appeared on camera. Acknowledgements were stamped with the wall clock, while the engine suppresses a notification whose send time falls after the acknowledgement, and every send time is a time on the scenario’s Tuesday. That compared two different calendars. It worked throughout development and broke silently the moment the real date passed the demo day, so explaining a finding greyed the row and suppressed nothing. Found it while recording the video, fixed it by stamping the household clock, which is what the test file had assumed all along.

Layout at real sizes. The floor plan is about 0.74 times its own width in height, so it can never fit a 16:9 window at full width. The top bar wrapped into an unreadable block at 375 pixels and the findings table was unusable there. The table now collapses to stacked entries on narrow screens and the bar splits into two rows.

Making a demo video with no borrowed material. Stock footage and music are exactly what the rules forbid and would have looked borrowed next to the product’s own typography, so the film had to be generated from the product itself.

Accomplishments that we're proud of

The system says “cannot tell” when it cannot tell. Toggling one charging window in the Ring simulator flips the afternoon walk between “cannot tell” and “did not happen” live, and the difference between those two outputs is the entire trust argument.

Nobody enters a schedule. Eleven routines, their windows, their confidences and their support all come out of the history on their own.

The evidence is checkable at a glance. “It has happened on 21 of 21 days” is a sentence you have to trust. A row of 21 marks, each offset by how early or late it ran and ending in one hollow one, is the same claim and you can audit it in a second.

A keypress that makes the argument. Press G and the afterimages disappear, leaving exactly what Ring recorded, which is everything any other system in this category can show you. That gap is the product.

The loop is closed. Thresholds are numbers next to names, so “Ruth is told at 92” is a sentence somebody can disagree with, which “smart notifications” is not.

46 assertions covering the learner’s negative controls, every verdict in the demo day, the coupling test, the coverage logic, off schedule handling, the escalation thresholds and their suppression, and the promise printed on every simulator button.

What we learned

A monitoring product’s quality lives in what it refuses to say. Almost all of the engineering here went into not making claims: rejecting formless patterns, admitting blindness, declining causal links that the data does not support.

Determinism is what makes a demo reproducible and a statistic honest. Seeding the generator and keeping the engine free of UI imports meant every number in the video could be verified by a reviewer running one command.

Filming a product is a test suite of a different kind. Driving every surface in sequence surfaced a real bug that months of clicking around had hidden, because it only appeared once the calendar moved past the scenario’s date.

Dark is the house style of this entire category, and a default is exactly what makes interfaces interchangeable. Building light by default, with dark tuned separately rather than inverted, cost more and was the right call.

Unknown has to be a type, not a convention. Once it is in the type system, every surface is forced to handle it, and the voice layer cannot quietly round it off to a yes or a no.

What's next for AFTERIMAGE

Ship reason.ts as a Streamable HTTP MCP server. It already returns the structured results such a server would serve, so what is missing is the transport, not the logic.

Pull real Ring history through the Ring APIs instead of the deterministic generator, and run the engine as a Lambda behind a Ring Edge rule so verdicts are computed where the events already are.

Package the Fire TV surface for Vega with a WebView wrapper and verify it on a Fire TV Stick 4K Select rather than at TV dimensions in a browser.

Learn across days rather than within one. The week strip already shows the six days before today, and the next step is a trend a family can act on, such as the dose drifting 20 minutes later each day for a week.

Import the care plan rather than hard coding it, so the consequence weights come from the agency or the clinician who actually knows what a missed dose costs.

Learn the acknowledgements. If the answer is always “the visit was rescheduled” on a Thursday, the system should stop asking on Thursdays.

Share this project:

Updates

Submission history