Inspiration Stroke rehab is repetitive and hard to stick with dropout rates high because patients lose motivation. Music is something almost everyone connects to, so we wanted to bridge that gap.
What it does EchoSphere turns rehab exercises into a rhythm based experience. Patients pick a song, and their prescribed movements sync to the beat. The app uses the device camera to track motion and gives real time feedback. Additionally, patients can check their post stroke condition with the use of a facial symmetry model that also uses mediapipe.
How we built it React/three.js frontend, mediapipe for hand & face movement detection, Node.js backend, and the Web Audio API for playback and beat syncing.
Challenges we ran into Pose detection was unreliable at first, especially for the slower, partial movements common in rehab. Syncing motion tracking to the beat was also tough latency stacks up across camera, processing, and audio output.
Accomplishments that we're proud of We built a working end to end prototype within the hackathon: open the app, pick a song, complete a guided session with live feedback. The UI stays approachable instead of clinical.
What we learned Pose detection is much harder in practice than in polished demos, especially for users with limited mobility. We also learned a lot about designing for accessibility our early prototypes assumed a range of motion many stroke patients don't have.
What's next for EchoSphere Consulting with physical therapists to ground the exercise library in real practice, expanding beyond upper body movements, adding long term progress tracking, and exploring Spotify integration.
Built With
- css3
- javascript
- mediapipe
- python
- webaudioapi
Log in or sign up for Devpost to join the conversation.