Inspiration
Gait, the way that humans walk is a critical measurement of overall health condition, efficacy of rehabilitation, state of disease etc. This measure is significant for neurological diseases such as Parkinson's Disease, and other pathologies. At the clinical level, the gold standard of gait assessment (or walking assessment) is done at the clinic using highly expensive tools, called gait carpets. Doing these assessments at the clinical level requires both time from patients and doctors to schedule, appoint, commute, complete, & analyze. Patients also tend to change the way they walk at the clinic (due to the white coat effect), which further implicates the external validity of gait assessments at clinical locations. The inspiration for this idea came from the thought of "What if we could complete a similar assessment at the home level that requires an absolute minimum level of interaction from the patients?".
What it does
We used computer vision to map the joints and segments of the lower body. Based on these mappings we created a system of gait event detection, specifically step events. Step events are the backbone of gait analysis, using these events we generate key metrics of gait analysis that are similar to the metrics extracted from traditional gait analysis methods using the gait carpet.
How we built it
We used Mediapipe and OpenCV in Python to do the computer vision task with a Logitech webcam attached to the computer. Since we stored the key metrics on gait, we also used the numpy and pandas packages in order to efficiently store data in CSV files.
Challenges we ran into
Making fine adjustments to webcam viewing angle and precise distance measurement to the body was critical in getting reliable data from the openCV pose tracker. Laptop based cameras were not sufficient and thus we had to grab an external webcam to do the video streaming. Using a singular camera also meant that the angle of video capture was also critical and we learned through trial and error that movement in just one plane is near impossible and thus moving slightly towards or away from the camera also impacted our vertical threshold calculations we used for step event detection (as the projection of the visual space is not exactly rectangular from the POV of the camera).
The project also went beyond just pose tracking and into step event detection. This meant using pose tracking and kinematics of human movement to mathematically create a step detection algorithm which combined time domain data analysis (using temporal aspects of a landmark's movement) and setting the precise threshold for footfall (stepping down) and foot-off (lifting the foot) detection. We initially opted to use joint kinematics but we found this to be a challenge due to the repeated joint movements and the variation in joint movements across subjects could possibly mean this system worked very poorly for people who did not move their legs as much.
Accomplishments that we're proud of
We were able to extract step time data from our step detection algorithm with values that were very similar to clinical level gait analysis. Considering how expensive a gait carpet can be (~$10,000 + the cost of space & time required to install + proprietary software requirement) we were proud to achieve one key metric in gait analysis that is easy to access and reproduce with consumer-grade technology.
What we learned
There is a reason this seemingly simple solution does not exist at the clinical level... yet. Humans have a hard time walking in a straight line, this creates multiple sources of error to deal with when trying to do step detection and even pose estimation. We also learned that laptop cameras are not it and a mounted webcam / camera works best for this type of detection. Learning from our teammates, we learned about the intricacies of applying computer vision to solve real world health issues, the wide field of gait analysis and how something as simple as walking can be a very useful tool to measure overall health of an individual. We also learned from each other in developing tools using previously novel frameworks and packages such as pandas & numpy.
What's next for Gait@Home
Polishing the metrics and making sure we collect more than one aspect of gait analysis is definitely the next step forward for this project. Ideally, we want to be able to collect almost all the types of data using this simple system that a traditional gait carpet collects. Reaching the critical "clinical standard" is also a goal for this project as we develop this tool further. Ideally a mobile app and incorporated phone camera system for the purpose of gait event detection is also a future goal. One important factor in this project is privacy, we could possibly seek external help and assistance in utilizing the perks of blockchain tech to preserve this patient - doctor and overall healthcare-related privacy.
Log in or sign up for Devpost to join the conversation.