Footsies

Footsie is a web tool designed to help users explore plantar fasciitis risk screening. Plantar fasciitis is the inflammation of the plantar fascia. Plantar fasciitis affects 1 in ten people.

Comment

Inspiration

Plantar Fasciitis is a chronic disease that affects 1 in 10 people. This disease results of inflammation of the thick band of tissue that connects the heel bone to the tones (Plantar Fascia).

During high school, one of our group mates (Rahul) had the opportunity to work with Plantar Fasciitis for his senior capstone research project. Being a competitive speedskater, his training was put on a halt when he started experiencing sharp pain in the soles of his feet. With nothing working after experimenting as well as visiting multiple specialists, he took this topic as his Senior project to understand the deeper mechanics of muscle sole inflammation.

For his project, he combined literature review (NLM) with 11 patient interviews under physician mentorship, and authored a meta-analysis on diagnostic strategies and pain-management outcomes.

What it does

The main objective of Footsies is when given a full body image where the feet are clearly visible, Footsies will be able to tell if ones foot is overpronated or not. Overpronation is the key indicator of Plantar Fasciitis. We analyzed overpronation by analyzing how much the heel eversion angle is (how far the heel is tilting outward relative to the lower leg).

Our Website asks users to video themselves walking. The users then take this video and find the exact frame of when the foot is maximally on the ground (the heel and toes fully lie flat on the ground). The users then take this frame (picture essentially) and submit that to the Footsies website which then outputs if the foot is pronated or not. Depending on the output, the website then prompts the user to a treatment page.

How we built it

The original vision of the project was for the user to take a video of them walking and for Footsies to fully be able to find the exact frame of when the foot is maximally on the ground (the heel and toes fully lie flat on the ground) and from there calculate if the foot is pronated or not.

With this vision we split our project into four parts. Front end Twelvelabs API usage Media Pipe Pose Usage Frontend to Backend connection

The front end is fully building our website using React (TypsScript). The Twelvelabs API (Python) was originally going to be used to find the exact timeframe of when the heel and the foot is maximally on the ground (the heel and toes fully lie flat on the ground). The Media Pipe API (JavaScript) would then find that frame and insert a skeleton to then run analysis to find the angle between the heel/ankle and ankle/knee. If the angle is above a certain threshold, we can then tell if the foot is overpronated or not.

Challenges we ran into

Although our original vision involved using TwelveLabs to find the exact timeframe of when the heel and the foot is maximally on the ground (the heel and toes fully lie flat on the ground), we quickly released that this was not possible using the TwelveLab models. We tried to then pivot using math and physics to find the exact frame to then then allow for the Media Pipe Pose skeleton analysis to run but this proved to be even more challenging due to perspective and camera issues that occur.

We originally thought we could incorporate Twelvelabs for trimming the video for only walking sections which ended up working (TwelveLabs API integration successful!). But in the end the analysis to find the exact frame was not possible.

After much trial and error, we defaulted to having the users send in a picture of their leg fully on the ground to Footsies because our Media Pipe Pose Skeleton analysis was extremely successful.

Accomplishments that we're proud of

Regarding the physical project, our group is extremely proud of our website design as well as our Media Pipe Pose Skeleton analysis. Even though the video aspect of the project did not pan out, our group is extremely proud of what we were able to accomplish. Two of this group's members have never competed in a Hackathon and found this process rewarding and valuable.

What we learned

Technical:

  • API Integration (Media Pipe & TwelveLabs)
  • How to use React to make a website from scratch
  • JavaScript, TypeScript, Python

Non Technical:

  • Team building/organization
  • Project building/organization
  • Communication

What's next for Footsies

  • We aim to fully implement this software in accordance with our original vision. We hope to be able to give Footsies a full video of someone walking and be able to fully see if that person is overpronating or not.

Built With

Share this project:

Updates