Inspiration

After orthopedic surgery or injury, recovery depends on home exercise programs done alone for weeks. Published adherence sits around 35–50%, and therapists only discover that a patient stalled, regressed, or started compensating at the next visit. Range of motion, the primary outcome, is still measured with a hand-held goniometer a few times a month. Nobody sees the other 95% of the reps. We wanted the laptop every patient already owns to become that instrument, and we wanted a therapist's judgment, not just a rep counter, to run after every session.

What it does

ArcRx turns a laptop webcam into a clinical range-of-motion instrument backed by an Entrinsik Informer App.

  • Measures every rep on the device. MediaPipe Pose and Face Landmarker models run in the browser (WebGL with CPU fallback, models served from the app). Shoulder abduction and elbow flexion use pose landmarks; cervical rotation and lateral flexion use the face model's head transform. Frames never leave the laptop; only scored angles are saved.
  • Scores movement quality, not just count. Each rep records peak angle, duration, end-range hold, a smoothness score, and compensation flags that therapists watch for: trunk lean, shoulder hike, shoulder drift, head tilt, chin drop.
  • Governs the data in Informer. Sessions and reps land in the app's private Postgres workspace. Patients see only their own record; clinicians see the caseload. Every server route enforces the role.
  • Lets an agent act. The rehab_triage agent runs on every session.completed event: it reads 30 days of history, assesses the trajectory (on track / ready to progress / plateau / regression / low adherence), raises alerts, adjusts the prescription within guard rails (±10°, ±2 reps), and writes a SOAP-style progress note. A weekly_caseload_report agent runs Monday 07:00 and flags adherence gaps for outreach.
  • Answers questions over the caseload. "Who regressed this week?" is answered by a model-drafted, validated read-only SQL query when a model is available, or by a deterministic intent router over the same tables, always scoped to the asker's role.
  • One-click Demo Simulation drives the identical pipeline with a virtual patient if lighting or camera access fails.

How we built it

Frontend: Vite, React 19, TypeScript, Tailwind v4, Recharts. A canvas overlay draws the skeleton and a goniometer arc with the target tick directly on the mirrored video. Computer vision: @mediapipe/tasks-vision Pose Landmarker (lite) and Face Landmarker. A 1.5-second calibration to the patient's own neutral posture, a One Euro filter, and a rep state machine (minimum range, return threshold) make counts and peaks repeatable. Compensation uses exercise-specific geometry (shoulder rise relative to torso length, hip–shoulder deviation from vertical, head roll during rotation). Informer App: informer.yaml declares roles, events and two agents; server/ holds file-convention routes mounted at /api/_server; tools/ holds the agent tools; migrations/ carries the schema plus six weeks of realistic demo data for four patients; lib/ holds shared domain logic. Everything conforms to the documented Informer Apps shape and deploys with npm run deploy.

Pitch Deck: https://bodapati-arcrx-rehab-coach.go.informer-apps.com/deck

Built With

Share this project:

Updates

Submission history