While we were researching potential hackathon ideas, one of our fellow lovely members stated that people who suffer from Bell’s Palsy may undergo functional electrical stimulation (FES) treatment to enable the patient to feel and move their facial muscles. These people as well are usually given physical therapy exercises to try and move their muscles on their own by physically moving their faces into different emotional states.

That’s when we realized something. Feelings can range from sad, afraid, happy, and excited. Our personal emotions allow other people to understand us better and learn ways to respond to our actions. However, around 40,000 people in the United States are affected by Bell’s Palsy yearly which prevents the patients from expressing their emotions through facial expressions. In addition, people who suffer from autism are unable to communicate their facial expressions as well. With these selected people in mind, our team aspires to combat this problem through technology.

What it does

Introducing TherAssist! It is a web app that detects the facial expression of the user based on their face using machine learning. This app can be used by occupational therapists or parents when guiding children who suffer from autism. Through this app, they can start teaching the children to evaluate, assess and associate their facial expressions with the proper emotions. TherAssist is a tool that recognizes a person’s emotions to educate the user of the proper term for their feelings.

Challenges we ran into

  • Our team faced difficulties in communicating with each other due to time zone differences. With the time zone difference, some of us had to compromise by sacrificing our sleeping schedules.
  • In addition, sometimes the internet connection sucks. It’s difficult to be working on a project during a call with a bad internet connection.
  • However, we did not want time zone differences, sleep deprivation and a bad internet connection to hold us back from participating at DefHacks 2020.

Accomplishments that we're proud of

  • Completing a project that is really applicable and solves real world problem -Learning about building apps in python -Learning about computer vision -Learning about machine learning and creating a working app that uses trained models

What we learned

  • Haar cascades are limited in what they are able to detect but run extremely faster than CNN models making it easier to make Haar cascades live which would be useful for webcam applications but you would still need to find a way to train them.

What's next for TherAssist

The next steps for TherAssist are adding features to allow different areas of study (eg. learning alphabets), generating performance reports by therapists to be transmitted to the parents, enabling therapists to assign homework to the patient and improving the UI design of the app.

Together, let’s feel what is real! Together, let’s feel the reality!

Built With

Share this project: