Therapy Hack

Combating the stigma around mental health

Comment 1

Inspiration

The need for mental health services is increasing, however the availability of these services is unable to meet the demand. We do understand that at the moment it is complex to completely automate the expertise needed to replicate an effective therapist, which is why our project "Therapy Hack" acts a supplement to aid mental health service by using sophisticated algorithms in Computer Vision and Deep Learning

What it does

  • The user enters their information like name, gender, address etc in a form which will be added to the patient database
  • The Questionnaire will ask the user a series of questions to know more about what they're going through
  • Then, the user uploads a video elaborating on themselves, which is then analyzed using Convolutional Neural Networks (CNN) and Vision API to detect the user's visual and vocal emotions
  • The user's audio is converted to text and the CNN performs sentiment analysis which outputs the likelihood of their emotions
  • The user's visual is analyzed by the Vision API to detect key facial landmarks to detect emotions like Happy, Sad, Surprised, Angry
  • A scheduling algorithm in Python sorts the doctors and patients depending on the user's address provided and gives the user a list of available doctors

How we built it

  • We developed a web application framework "R Shiny" as a prototype of user interface which uses R as back-end and Shiny as front-end with Google cloud storage API
  • The text analysis is done using Convolutional Neural Networks (CNN) by exploring various models offered by pytorch. The decision is made by converting words to vectors. These mappings are then used to label with the sentiments based on the training data labels.
  • We also used Python to input the user data obtained from the Questionnaire, Text analysis and Facial Recognition. This data is then used to schedule doctors for an appointment on suitable date preferred by the user by giving weightage to the scores obtained from Questionnaire, Text analysis and Facial Recognition. This algorithm uses Geocoding API and Distance Matrix API from Google cloud platform
  • Used Vision API to detect key facial landmarks

Challenges we ran into

  • I (Ponaravind) used APIs for the first time so it took a while to get familiar with it and incorporate in the algorithm. It was challenging to work on the scheduling algorithm to schedule patients corresponding to the location, availability of doctors and score obtained from the Questionnaire, Text analysis and Facial Recognition
  • I (Harshitha) worked with text sentiment analysis. The challenging part was to be able to get the right model and dataset to train and to handle the complexity within the given time-frame, keeping the accuracy in mind

Accomplishments that we're proud of

  • Planning and delegating work appropriately to each team member right from the beginning and sticking to the deadlines.
  • Integrating various modules into a single customer experience

What we learned

  • We learnt how to use R Shiny, how to integrate different programming languages
  • Training different kinds of models and understanding relation between different processes
  • Using APIs to communicate between different models

What's next for Therapy Hack

The next step is to increase the accuracy and reliability of the model. This could be done by using a research based questionnaire. The decisions made by the text and video analysis modules could be optimized by research results that tell us about a person's behavior based on their mental health. This could then be used as a more approachable solution for people seeking help

Built With

Share this project:

Updates