movement analysis for progress checker
can your wheelchair pass?

Inspiration

Our inspiration came from people close to us and some of the difficulties they face in everyday life. We wanted to build something that could genuinely help people recovering from spinal cord injuries, while also being realistic to use outside of a hospital or rehabilitation centre.

One of the main problems we noticed is that progress during rehabilitation can be difficult to measure. Improvements can happen slowly, and it is not always easy for a patient to notice whether their movement is actually getting better. We wanted to see if we could use something as simple as a phone camera to make this progress easier to track.

At the same time, recovery is not only about exercises. People with limited mobility also face problems with accessibility and managing their daily routine. This is why SpyMotion eventually became more than just movement tracking.

What it does

SpyMotion is a privacy-focused platform that helps people recovering from spinal cord injuries track their movement progress and deal with some everyday accessibility challenges.

Progress Tracking

The main feature of SpyMotion is movement analysis.

A user can film themselves performing one of the supported rehabilitation exercises and upload the video. SpyMotion uses pose estimation to track the relevant joints throughout the movement and calculate the range of motion.

Instead of only giving one final score, we try to show information that is actually understandable. This includes the peak range of motion, the range achieved during repetitions, how consistent the movement was, and how the result compares with previous recordings.

This makes it possible to see whether movement is improving over time instead of relying only on how the exercise feels on a particular day.

We also wanted the system to be honest when it does not have enough information. Pose estimation can fail because of poor lighting, the camera angle, part of the body leaving the frame, or other interference. If too few reliable frames are detected, SpyMotion does not try to guess a result. Instead, it tells the user that the video cannot be scored reliably and explains how they can record it again.

Access Scan

We also created Access Scan to help with a different problem faced by wheelchair users.

The user first enters the width of their wheelchair and the height of the armrest. They then take a photo of a passage while holding the phone upright at approximately armrest height.

SpyMotion analyses the image and estimates whether the available space is wide enough for the wheelchair to pass through.

The result is intentionally kept simple. The user receives either Can pass, Cannot pass, or Unable to determine.

We included the last option because we did not want the system to give a confident answer when the image does not contain enough information to make a reliable estimate.

Medication Reminder

SpyMotion also includes a medication reminder feature.

The idea behind this was to keep more of the user's rehabilitation routine in one place. Someone can use SpyMotion to track their physical progress while also keeping track of medication they need to take throughout the day.

Although movement tracking is still the main focus of SpyMotion, we see this as one way the platform could eventually become a more complete rehabilitation companion.

How we built it

SpyMotion was built mainly in Python, with AI tools also helping us during development and debugging.

For Progress Tracking, we use MediaPipe and BlazePose to identify body landmarks in each frame of a video. From these landmarks, we calculate joint angles using vector geometry and analyse how those angles change throughout the exercise.

The video therefore goes from pose estimation, to joint coordinates, to a time series of angles, and finally to measurements such as range of motion and repetition consistency.

We also analyse the quality of the detected landmarks. This is important because a movement measurement is only useful if the body was tracked properly in the first place.

Access Scan uses a separate computer vision approach. It uses information from the image together with the wheelchair dimensions entered by the user to estimate the available width of the passage.

The interface was built with Streamlit. We created both a normal web layout and a mobile-focused layout so that SpyMotion can also be demonstrated and used from a phone. We also added dark mode, progress graphs, previous session comparisons, skeleton visualisation and instructions to help users record videos correctly.

Another important part of the project was keeping the system local-first. SpyMotion does not require an account, API key or cloud database to analyse rehabilitation videos. We wanted sensitive videos to remain under the user's control rather than being uploaded somewhere unnecessarily.

Challenges we ran into

One of the biggest challenges was combining several different ideas into one application.

Progress Tracking and Access Scan both use computer vision, but technically they are very different problems. One focuses on tracking a person's movement across many video frames, while the other tries to estimate physical space from a single image. Getting both of them working inside the same application took quite a lot of integration.

Another challenge was making the movement analysis reliable. It is quite easy to calculate an angle once landmarks have been detected. The difficult part is deciding whether those landmarks are actually good enough to trust.

During testing, we saw how much lighting, camera positioning and occlusion could affect the result. Because of this, we added checks for tracking quality instead of always forcing the system to produce a score.

We also spent a lot of time on the interface. Raw joint coordinates and computer vision outputs are not very useful to most patients, so we had to think about how to turn them into information that someone could actually understand.

Finally, the main challenge was simply time. We had to build the computer vision pipelines, connect them to the interface, test them, improve the design and make the different features work together within one hackathon.

What we learned and accomplishments that we're proud of

One of the biggest things we learned was that building something with AI for healthcare is not only about getting the model to work.

It is also important to think about when the model might be wrong, how that uncertainty should be shown to the user, and whether the result is actually useful.

For Progress Tracking, we are proud that we managed to go from a normal video to actual movement measurements. We implemented pose estimation, calculated joint angles, detected repetitions and used this information to show how movement changes over time.

We are also proud of the reliability checks. Instead of always showing a result, SpyMotion can recognise when the tracking quality is too low and refuse to give a misleading measurement.

For Access Scan, we managed to use monocular computer vision to approach a completely different accessibility problem within the same project.

We also created a small validation set for the movement analysis. For each of our four exercises, we recorded a normal movement, a movement with deliberately reduced range of motion, and a recording with interference. This gave us 12 videos that we could use to check whether the system reacts in the way we expect.

Another accomplishment was getting everything into one interface that works both as a normal website and in a phone-focused view, including dark mode and progress visualisation.

Responsible use of data

Because SpyMotion deals with rehabilitation videos, we thought carefully about what data we actually need.

The system is designed so that movement analysis can happen locally without requiring a cloud account or external API. We do not need to build a central database containing patient videos just to calculate movement measurements.

We also deliberately avoid presenting SpyMotion as a diagnostic system. It is designed to help users track their own movement progress, not to diagnose spinal cord injuries or replace a doctor or physiotherapist.

This also influenced how we handle uncertainty. If the system does not have enough reliable information, we believe the safest result is to say that we cannot determine the answer rather than showing a number that looks more certain than it really is.

Impact

The part of SpyMotion that we find most exciting is that it does not require specialised motion capture equipment.

The goal is to make useful movement tracking possible using technology that many people already have access to.

Someone recovering at home could record the same exercise every few days and see whether their range of motion is changing. Instead of relying only on memory or how the exercise felt, they would have an objective measurement that they can compare over time.

In the future, this information could also be useful when discussing recovery with a physiotherapist or doctor.

Access Scan follows the same idea. Rather than requiring special equipment to measure every unfamiliar doorway or passage, we wanted to explore whether a phone could help give users more information before they try to pass through it.

For us, the main goal of SpyMotion is not to replace healthcare professionals. It is to give patients more information and independence between appointments.

What's next for SpyMotion

The biggest next step would be improving and properly validating the accuracy of our measurements.

For Progress Tracking, we would like to compare SpyMotion's calculated joint angles with measurements taken using established clinical methods. We would also like to test it with more people, more exercises, different camera positions and different levels of mobility.

Access Scan could also be improved by using better calibration and more advanced depth estimation.

We would eventually like to turn SpyMotion into a real mobile application. This would make recording exercises and using Access Scan much more natural than through a web interface.

Most importantly, we would want to continue developing the project together with people who actually have experience with spinal cord injury and rehabilitation. We built the current prototype based on the problems we understood during the hackathon, but the best way to improve SpyMotion would be to learn directly from patients and clinicians about what would genuinely be useful to them.