Inspiration
Physical therapy and clinical musculoskeletal rehabilitation rely heavily on continuous biomechanical feedback[cite: 17]. However, millions of patients recovering from orthopedic injuries, post-surgical repairs, or chronic joint degradation face high clinic fees, limited therapist availability, and a lack of objective guidance during home exercise[cite: 17].
When patients perform prescribed rehabilitation routines unsupervised, minor postural deviations, improper flexion depth, or compensatory loading often go unnoticed[cite: 17]. Over time, this leads to secondary joint strain, compensatory imbalances, and elevated reinjury rates[cite: 17].
We built RehabOptics to transform standard monocular webcams into markerless biomechanical tracking systems[cite: 17]. The platform delivers instantaneous kinematic corrections and quantified motion metrics on edge devices without requiring motion-capture suits, wearable sensors, or clinic visits[cite: 17].
What It Does
RehabOptics is a low-latency edge computer vision platform that computes planar joint angles in real time[cite: 17]:
- Markerless Joint Localization: Tracks key lower-limb anatomical nodes (hip, knee, and ankle) frame-by-frame via standard optical video input.
- Dynamic Planar Trigonometry: Computes continuous interior joint angles at 70+ FPS on standard consumer hardware.
- Phase Segmentation & Threshold Verification: Automatically counts valid repetitions, verifies eccentric depth against therapeutic targets (e.g., flexion <= 95°), and flags incomplete cycles.
- Heads-Up Telemetry HUD: Visualizes live joint angles, movement phases (descent vs. ascent), and rep counts on an adaptive overlay.
- Automated Clinical Summaries: When a session concludes, the engine synthesizes an objective rehabilitation report breaking down Range of Motion (ROM), depth compliance, and actionable kinematic adjustments for physical therapists to review.
How We Built It
- Frame Capture & Preprocessing: The video pipeline processes standard webcam input using OpenCV, applying contrast-limited adaptive histogram equalization (CLAHE) to maintain tracking stability across variable room lighting.
- Pose Detection & Joint Localization: Deep convolutional landmark architectures map spatial (x, y, z) coordinates along the lower extremity kinematic chain.
- Kinematic Math Engine: Dynamic vector geometry computes joint angles across the hip (A), knee (B), and ankle (C):
- Direction Vectors: Vector u = A - B and Vector v = C - B
- Interior Joint Angle: Calculated using the normalized dot product of vectors u and v with arc cosine transformations: θ = arccos((u · v) / (|u| |v|))
- Frontal Plane Symmetry: Analyzes lateral displacement ratios to identify dynamic knee valgus and asymmetric loading patterns.
- Automated Diagnostic Reporting: Angular time-series metrics are processed through the Google GenAI SDK to generate structured clinical evaluations with built-in multi-model fallback handling for uninterrupted reliability.
Challenges We Ran Into
- Throughput Optimization: Maintaining 70+ FPS while calculating trigonometry on every frame required stripping out CPU-bound image transformations and streamlining vector operations in NumPy.
- Real-Time Phase Calibration: Accounting for anatomical variations and noise near inflection points required implementing temporal smoothing buffers to prevent false rep counts.
- API Availability & Fault Tolerance: Cloud endpoints occasionally experience temporary surges or deprecations. We engineered a multi-tier fallback mechanism across candidate models alongside deterministic local telemetry backups to guarantee zero-downtime report generation.
Accomplishments That We're Proud Of
- Achieved sub-15ms edge processing latency (consistently exceeding 70 FPS) on standard laptop hardware without needing external acceleration.
- Built a markerless computer vision pipeline that computes reliable kinematic angles from a single webcam feed.
- Created an end-to-end user workflow: from live visual guidance during exercise to clinician-ready reporting upon session exit.
What We Learned
- How to structure real-time video frames for low-latency mathematical computations without dropping display frames.
- Practical vector biomechanics, specifically translating joint coordinate points into clinical kinematic data.
- Techniques for architecting robust API fallbacks to ensure client applications degrade gracefully during upstream outages.
What's Next for RehabOptics
- Multi-Planar Tracking: Expanding from 2D planar tracking to full 3D spatial reconstructions to assess frontal and transverse plane motion simultaneously.
- Multi-Joint Rehabilitation Chains: Expanding support to upper-limb kinematics (shoulder abduction, elbow flexion) and cervical spine posture.
- Clinician Cloud Dashboard: Enabling real-time session telemetry uploads to secure therapist portals for longitudinal recovery tracking and dynamic prescription updates.
Log in or sign up for Devpost to join the conversation.