ITap

ITap

70% of hand therapy patients fail at-home regimens because they lack the ability to measure 2-3 degree range-of-motion improvements. Users may struggle to see changes in movement, notice compensatory motions, or share useful progress data with clinicians.

ITap is an ML assisted physical therapist that uses computer vision to judge your exercise form, and uses a novel spatial keyboard feature for transforming a standard laptop into an interactive physical therapy workstation. ITap guides users through calibrated sessions with real-time feedback.

ITap is a rehabilitation-inspired prototype designed to process camera and microphone data locally. It features:

Live Digital Goniometry: Google MediaPipe tracks 21 hand landmarks to estimate finger joint flexion angles and range of motion in real time.
Movement Guardrails: A deterministic vector-math engine that measures changes in palm orientation from a calibrated baseline, pausing target scoring and displaying feedback when deviation exceeds the configured threshold.
Vibro-Acoustic Sensor Fusion (fancy talk for a spatial keyboard): Matches visual fingertip motion with microphone-detected impulses through the Web Audio API to measure desk-tapping speed, rhythm variability, and changes in tapping rate. Primarily used for hand exercises for Arthritis and Parkinson’s patients.