JSA's Fine Motor Skills app

Inspiration

Stroke is one of the top causes of death.
Every 40 seconds, someone gets a stroke, and someone dies from it every 4 minutes in the United States. Besides, patients who are recovering from a stroke usually have paralysis. The physical therapy could go up to $55 with insurance and $150 without insurance every time.
It might be difficult for some people to get treated every time at physical therapy. But people still need consistent treatment. Also, the number of physical therapy facilities is limited. Therefore, there is a need for an effective home physical therapy environment. But there are few problems doing home physical therapy as well. First, Patients may not receive accurate feedback. Since there is no professional coach beside you, so the activities might be less effective. Second, Repetitive exercises are hard to keep up and easy to get unmotivated.

But don't worry! Our web-based application solves these problems.

What it does

Our app is an online platform that you can have physical therapy. This image recognition based application will provide you with more accurate feedback and motivation to continue your physical therapy. Our app has variable interactive activities that would make patients work consistently.

An example of the activities is "grabbing a ball." The app detects a person's hand movement, and it let you grab and move the object on the screen.

As long as you are holding it correctly, the virtual object will move along with the person's hand.

DEMO video: https://youtu.be/6aicmhvOohs

How We built it

We have used several external technical resources, such as : ・The hand tracking system by Tensorflow, which contains the image recognition model developed by MediaPipe. ・Miro to brainstorm ideas. ・Figma for design prototyping. ・Vue.js is one of the modern frameworks for javascript, which is relatively easy to get started with.

Challenges We ran into

Since some teammates live in different time zones/countries and all of the meetings were online, it was sometimes hard to communicate the nuance of thoughts. Also, some teammate should accommodate their time to others.

Accomplishments that We are proud of

Although we pivot our project at the halfway of the hackathon, we were able to complete the demo code.

What We learned

We learn how significant team-building and communication are.

What's next for Trick or Treating Simulator

Some improvements that we could have: ・Real time feedback ・sound effects ・variable object setting ・variable type of exercises such as using more body parts and more motions ・and add entertaining aspects like stages and levels.