REWIND

Inspiration

Computers let us save, compare, undo, restore, and inspect history. Physical spaces do not.

REWIND asks: what if a room could have a checkpoint?

Instead of treating a camera only as a security sensor or video archive, REWIND turns visual observations into structured physical state. A user can save a space, let reality change, ask what changed, and receive step-by-step guidance until the saved state is restored.

What it does

REWIND implements SAVE → DIFF → REWIND → VERIFY.

SAVE captures a visual observation, converts it into semantic state, and persists a checkpoint. DIFF compares a fresh observation with that checkpoint using deterministic TypeScript. REWIND generates ordered human restoration actions from confirmed semantic differences. VERIFY re-observes the scene and recomputes progress until no meaningful differences remain.

Humans remain the actuator. REWIND supplies perception, memory, comparison, guidance, and verification.

How we built it

REWIND's primary Ring integration uses the Ring Developer Playground and WHEP live video:

Ring Playground → WHEP live frame → Amazon Bedrock / Nova 2 Lite → Physical State Protocol → deterministic diff → restore planner → fresh verification.

The public submission adapter provides a repeatable Image + REWIND Voice version of the same restoration loop inside one coherent product UI. The separately implemented Alexa Custom Skill remains a real external voice integration through AWS Lambda and the same deterministic REWIND engine.

The core architecture rule is:

AI interprets state. Deterministic code compares state.

Nova never decides whether the environment is restored. Deterministic code owns comparison, match scoring, restore planning, action verification, and the final RESTORED state.

AWS components include Amazon Bedrock + Nova 2 Lite, DynamoDB, Strands Agents SDK, Amazon Bedrock AgentCore Memory, and AWS Lambda.

Reliability work

REWIND refuses to interpret a vision model's omission as proof of removal.

For tracked checkpoint objects, a missing entity stays UNKNOWN unless focused contrastive Nova audits produce explicit high-confidence evidence that the saved support area is visible and the object is absent.

The final production removal regression removed two tracked notebooks simultaneously. REWIND produced two explicit absent states, two deterministic restore actions, partial verification, and final restored verification.

A second unrelated kitchen regression tested the opposite direction: an empty kitchen checkpoint was compared with the same scene plus a cup and kettle. Independent audits voted on the exact current candidate keys, both extras became semantic movable/present evidence, deterministic DIFF produced two ADDED changes, REWIND Voice guided cup then kettle, and a third next-step request correctly stopped instead of repeating the last action. The production logic remains object-agnostic; those kitchen nouns are test/demo evidence, not hardcoded behavior.

Ring track proof

The repository and live testing include real Ring runtime integration: device discovery, WHEP live view, frame capture, Nova semantic observation, Ring-backed save/compare, signed motion webhook verification, and Ring-backed agent/MCP operations.

The final video will show the Ring Developer Playground path directly.

Challenges

The Ring Playground does not expose deterministic physical rearrangement controls. We solved this by keeping evidence boundaries explicit:

Live Ring proves the real Ring transport/perception path. Image + REWIND Voice proves the repeatable production restoration interaction inside the web product. Controlled fixtures prove deterministic behavior under known ground truth.

A second challenge was negative visual evidence. The final design keeps omission UNKNOWN and requires explicit high-confidence consensus before absence becomes deterministic truth.

What we learned

The useful abstraction is not object detection. It is physical state.

Once a physical environment is represented semantically, software can reason about checkpoints, differences, restoration actions, progress, and verification without pixel-perfect matching.

We also learned to separate probabilistic perception from deterministic truth. Nova is excellent at interpreting visual context, but a restoration product needs auditable logic around what counts as a change and what counts as restored.

What's next

Multi-camera checkpoints, richer cross-view identity, production multi-user spaces, history/timeline UX, public Alexa+ integration when developer onboarding is available, and a standalone reusable Physical State Protocol package.

Privacy principle: Remember the state. Forget the footage.

Built With

• alexa-skills-kit
• amazon-bedrock
• amazon-bedrock-agentcore-memory
• amazon-dynamodb
• amazon-nova-2-lite
• aws-lambda
• model-context-protocol
• node.js
• ring-developer-apis
• ring-developer-playground
• strands-agents-sdk
• typescript
• vercel
• webrtc
• whep