Inspiration# STEADY MotionOS — The Trust Layer for Movement

Don't just count movement. Prove the measurement.

Inspiration

Twelve repetitions. One confident-looking number. But what if the camera couldn't actually see the person's legs?

That number might be correct. It might be wrong. Without reliable observation, the application cannot establish which.

This became the question behind STEADY MotionOS:

What if movement intelligence could recognize the limits of its own evidence—and actively help the user recover a trustworthy measurement?

Camera-based fitness applications often focus on recognizing poses and counting repetitions. But a standardized movement assessment needs more than an impressive counter. It needs consistent timing, observable movement phases, protocol compliance, uncertainty handling and an explanation of how the result was obtained.

For someone performing a chair-stand assessment with a helper, an unjustifiably confident result can be more misleading than an honest UNSCORABLE outcome.

We wanted to transform the largest screen in the home from a passive fitness display into an accessible, evidence-aware movement interface.

Our vision was simple but technically ambitious:

A system that observes movement, questions unreliable evidence, helps restore observation quality, and makes every accepted measurement explainable.

That's STEADY MotionOS.

What it does

A movement-intelligence operating experience built around six interconnected innovations

STEADY uses a smartphone as an on-device pose sensor and a Fire TV application as the large-screen guidance, assessment and evidence interface.

Rather than streaming raw camera footage to a server, the phone processes movement landmarks locally and sends structured measurement events.

The product's initial standardized workflow is the 30-Second Chair Stand.

01 — MotionProof: An Explainable Temporal Movement Engine

Not every upward movement is a completed stand.

MotionProof is a deterministic temporal state machine that reasons about movement over time.

It distinguishes seated reference, rising, standing and lowering phases, checking whether a repetition satisfies the movement rules.

Its standardized assessment workflow includes:

Phone-local 30-second timing
Full-stand validation and stable phase detection
Incomplete movement rejection
Assessment-boundary and halfway-rule handling
Supervisor-confirmed arm-assistance invalidation
Separate practice and standardized-assessment modes

Instead of just incrementing a counter, MotionProof retains why a repetition counted and which movement evidence supported it.

02 — PoseTrust: The Reliability Engine That Can Refuse to Guess

The central innovation isn't producing a count. It's knowing when the count is insufficiently supported.

PoseTrust evaluates three different questions:

Observation quality: Could the camera actually see the relevant body landmarks?

Protocol validity: Was the assessment performed under the required rules?

Measurement reliability: Are the observed movements sufficiently reliable to justify the reported result?

The system can distinguish a fully supported count from a result requiring review or an unscorable assessment.

For example, instead of silently reporting 12 repetitions after losing important landmarks, STEADY can report that 11 were supported and an additional movement needs review.

Confidence is not manufactured merely to produce a pleasing number.

03 — Active Observation Recovery + NOVA: A System That Helps Fix Its Own Evidence

STEADY doesn't simply show “Camera error.” It attempts to diagnose what went wrong.

When observation quality degrades, Active Observation Recovery evaluates the failure condition and selects a specific corrective instruction.

Examples include:

Feet or ankles outside the frame
Knees obscured
Poor lighting
Unstable camera position
Insufficient movement visibility

NOVA, our original event-driven companion, translates these technical conditions into plain-language, on-screen guidance.

If the user's legs are not adequately visible, NOVA suggests repositioning the camera rather than attempting to infer the missing movement.

After the camera changes, STEADY re-evaluates observation quality and acknowledges meaningful improvement.

NOVA is designed to respond to verified system events and stay quiet during the standardized assessment—not generate distracting, ungrounded commentary.

The result is a distinctive closed-loop interaction:

OBSERVE → DIAGNOSE → GUIDE → REOBSERVE → VERIFY

04 — MotionTwin: Phase-Aligned Movement Comparison

Two sessions with the same repetition count can still contain different movement patterns.

MotionTwin compares movement signatures using constrained Dynamic Time Warping (DTW).

It aligns corresponding phases of repeated movement, examining features such as rise duration, standing duration, lowering duration and trunk inclination.

But comparison is only useful when the underlying assessments are genuinely comparable.

The backend therefore verifies reference ownership, eligibility, provenance and protocol compatibility.

Synthetic recordings cannot silently become a person's LIVE baseline.

Incomparable assessments cannot masquerade as evidence of progress.

A measured difference is reported as a movement-pattern difference—not automatically interpreted as a change in health.

05 — Counterfactual Reliability Lab: Red-Teaming the Measurement

Instead of asking judges to trust our algorithm, we built a way to challenge it.

Our Counterfactual Reliability Lab systematically perturbs recorded landmark sequences.

It can simulate:

Knees hidden · Lower-body occlusion · Dropped frames · Coordinate noise · Timestamp jitter · Camera displacement · Reduced sampling frequency

The same underlying movement is processed before and after each perturbation.

The interface shows whether the engine:

Preserved the correct measurement
Appropriately withheld an unreliable result
Became uncertain
Or produced a confident error

This lets users and evaluators inspect the actual trade-off between measurement availability and reliability.

In our synthetic ablation, the final temporal PoseTrust configuration produced zero confident wrong counts across 88 deliberately perturbed recordings, compared with 26 confident wrong counts without PoseTrust.

That improvement came with substantial abstention: 77.3% of the perturbed recordings were withheld. We report that limitation rather than hiding it.

These are synthetic stress-test results, not real-human accuracy claims.

06 — EvidenceGraph: Every Count Has a Reason

A number should come with a measurement receipt.

EvidenceGraph connects raw landmark-derived observations, movement decisions, assessment validity, reliability outcomes, explanatory statements and companion actions.

A result isn't just a score.

It includes the evidence supporting the score, the relevant quality and protocol decisions, and the uncertainty that remained.

The backend independently validates impossible or contradictory results before accepting them.

If a correctly flagged arm-assistance review is later confirmed, the assessment can be superseded through a controlled, auditable correction. The invalid result cannot continue functioning as a valid personal baseline.

STEADY's defining idea: Measurement integrity should be enforced throughout the system—not added as a confidence label after the result is calculated.

How we built it

One shared movement engine. Two coordinated devices. A persistent evidence layer.

We designed STEADY as a modular, cross-device system.

Phone — On-Device Computer Vision

The phone application uses TypeScript, Vite and MediaPipe Pose Landmarker, running through a Web Worker with WebAssembly-based inference.

The smartphone processes camera frames locally and emits structured pose and assessment events.

A local assessment clock preserves standardized timing independently of television-network delays.

An at-least-once event outbox supports reconnection and reliable delivery.

Shared Core — MotionProof, PoseTrust and MotionTwin

A shared TypeScript engine implements temporal movement analysis, observation reliability, protocol decisions, recovery logic, phase-aligned comparison and counterfactual perturbation experiments.

The same core is reused across the phone application, TV-side analysis and Node-based evaluation.

This reduces the risk of different versions of the algorithm silently disagreeing.

Backend — FastAPI, WebSockets and SQLite

The service coordinates device registration, single-use QR pairing, on-TV approval, ordered events, de-duplication and persistence.

Assessment results are validated and durably saved before being presented as saved.

The server enforces source provenance, comparison eligibility, session ownership, command idempotency and bounded landmark uploads.

EvidenceGraph stores the relationship between observations, decisions, explanations and corrected results.

TV — Kotlin, Android WebView and Native Interaction

The Android TV application uses a Kotlin launcher shell hosting a bundled TypeScript TV interface.

It supports D-pad and Back navigation, native Text-to-Speech integration, service discovery and the accessible NOVA-guided workflow.

The Android APK has been compiled and signature-verified.

Evaluation — Synthetic Fixtures and Reproducible Failure Injection

We built a reproducible evaluation environment with annotated synthetic scenarios, reliability ablations, temporal movement perturbations, MotionTwin consistency checks and end-to-end browser tests.

The v0.3.3 release report documents:

90 Python tests + 63 Vitest tests + 31 Kotlin/JVM tests = 184 passing automated tests.

It also documents 28/28 browser end-to-end checks, including pairing, real MediaPipe inference on a static photograph, camera-view recovery, a simulated 30-second assessment, a forced network outage and saved-result verification.

These checks establish substantial software functionality, but desktop Chrome is not a verified Fire TV runtime.

Challenges we ran into

Challenge 1 — Confidence Filtering Initially Made the System Worse

One of our most revealing experiments showed that a simple confidence threshold could increase confident counting errors.

In the synthetic perturbation ablation, a confidence-only configuration produced 33 confident wrong results, compared with 26 without the reliability layer.

The reason was that confidence of individual observations was not enough to establish temporal consistency.

We needed evidence that the movement sequence remained coherent.

That motivated the stronger PoseTrust architecture, combining observation quality with temporal corroboration and protocol-level checks.

Lesson: A confidence score is not the same as reliable measurement.

Challenge 2 — Camera Movement Fooled Per-Frame Checks

A displaced camera can change the apparent body geometry without reflecting genuine human movement.

Earlier checks failed to adequately distinguish camera displacement from movement-pattern changes.

We introduced stronger reference and temporal-consistency checks so severe camera disruption could trigger an unscorable decision rather than a misleading count.

Challenge 3 — Reliable Results Across Network Interruptions

Phone-to-TV communication can be interrupted during an assessment.

We separated the phone-local assessment clock from network transport and implemented an event outbox, ordering, duplication protection and backend persistence checks.

Our browser-based end-to-end test deliberately interrupted the connection and verified that the synthetic assessment completed without duplicate result persistence.

Challenge 4 — Correcting an Assessment Without Erasing Its History

A late supervisor-confirmed arm-use event exposed an important integrity problem: accepting an invalidation means little if the original scored result remains authoritative.

We developed a controlled supersession workflow that preserves the original event, updates the authoritative result, revokes an invalid reference and invalidates affected comparisons.

That made protocol corrections part of EvidenceGraph rather than silent database overwrites.

Challenge 5 — Building the Android Application Under Toolchain Constraints

Our build environment could not reach some standard Android dependency repositories.

We created a reproducible command-line APK build workflow using the available Android SDK tools and Kotlin tooling.

The resulting Android package was compiled and signature-verified.

However, our environment lacked the virtualization and downloadable system images needed for a genuine Android TV emulator run.

We disclose that limitation rather than claiming browser screenshots prove Fire TV execution.

Accomplishments that we're proud of

1. We built a measurement system that can abstain.

STEADY distinguishes a completed movement from a visible but incomplete movement, an inadequately observed movement, an invalid protocol run and a supported measurement.

2. We created active recovery rather than passive error messages.

NOVA responds to observation failures with specific corrective guidance and acknowledges when conditions improve.

3. We demonstrated substantial reliability improvement on deliberately corrupted synthetic sequences.

In the 88-recording synthetic perturbation ablation:

No PoseTrust: 26 confident wrong outcomes
Temporal PoseTrust: 0 confident wrong outcomes
Cost of the conservative policy: 77.3% abstention

We treat abstention as a measurable trade-off, not a hidden success metric.

4. We built a reproducible Reliability Lab.

Users can observe how camera occlusion, missing frames, noise and displacement affect the same underlying assessment.

5. We implemented phase-aware MotionTwin comparisons.

The system investigates movement-phase differences while enforcing baseline eligibility and comparison compatibility.

6. We built end-to-end result integrity.

Valid result persistence, malformed-event rejection, reconnect recovery, command idempotency and auditable assessment correction are enforced beyond the frontend.

7. We packaged an Android TV application.

The repository includes a Kotlin Android TV shell, bundled UI assets and a compiled, signature-verified APK.

8. We engineered and documented 184 automated test passes.

Our release report includes 90 Python, 63 JavaScript and 31 Kotlin/JVM passes, alongside a separate 28-step headless-Chrome end-to-end demonstration.

Our proudest accomplishment isn't the highest possible repetition count. It's building a system designed to recognize when that count cannot be trusted.

What we learned

STEADY changed how we think about AI-driven measurements.

A correct answer without reliable evidence may be indistinguishable from a lucky guess.

A good movement system needs more than pose estimation. It needs temporal rules, observation-quality diagnostics, protocol validation, provenance, repeatable evaluation and a method for correcting authoritative history.

We learned that:

Uncertainty is a product feature. Users deserve to know when a result needs review.

Recovery is more useful than a generic failure. A specific, actionable camera instruction can restore the conditions needed for measurement.

Network reliability and measurement reliability are different problems. Both must be addressed independently.

Comparability is not automatic. Movement histories should not be compared unless their reference data is genuinely compatible.

Synthetic evidence has limits. Strong results on controlled perturbations do not establish performance on real people, phones or living-room environments.

The most important design principle we took away was:

A trustworthy AI product must be able to justify not only what it reports, but also what it refuses to report.

What's next for STEADY MotionOS

1. Genuine Fire TV verification

Install and test the packaged application on a supported Android TV emulator or physical Fire TV, including D-pad navigation, native speech, pairing, reconnection and performance.

2. Real smartphone and movement validation

Collect safely supervised, consented and manually annotated chair-stand recordings to quantify real movement-detection error, observation failures and abstention behavior.

3. Improve the reliability–availability trade-off

Use real recordings to reduce unnecessary abstentions without reintroducing confident wrong results.

4. Multi-profile household support

Extend the current single-person-per-TV prototype with explicit separate measurement histories and baselines.

5. More accessible movement workflows

Explore additional carefully defined movement tasks only after the foundational chair-stand workflow is validated.

Our longer-term goal is to make the large-screen home experience a better place for understandable, accountable movement technology.