Body Frame

Let us detect your mood and diseases to make you feel better.

Comment

Inspiration

Not everyone has the right state of mind and body in this pandemic but almost everyone has a webcam.

In a digital world affected by COVID-19, digital diagnosis is more necessary than ever. The pandemic has caused an influx in hospital cases, and the limitations on hospital beds have people wondering whether their symptoms are severe enough to warrant a doctor's appointment. Meanwhile, others experience ailments but are unable to afford a visit to the doctor due to a lack of or poor health care. We are in alarming need of free and practical digital healthcare for keeping in touch with our medical needs. More than 2 in 5 people globally reported symptoms of depression and anxiety during the pandemic. As students, we have realized that a lot of our time is spent in front of a laptop and first-hand watched our mental health deteriorate during the pandemic. That’s why having an application like BodyFrame is so important!

What it does

BodyFrame is an app that allows users to regularly check up on their physical and mental well-being without needing to visit a doctor.

We analyze your health using just two inputs - your video and your text. The features of our product are numbered below:

  1. First, it can register the "mood" of a person. This is very important to provide automated action and suggestions from the website and to the user. The product can detect a total of 4 moods: happy, sad, angry, and neutral through the webcam of the user. With different moods, the website can perform different actions. If the user is happy, the website suggests if it should play the user's favorite Spotify playlist. If the user is sad, the website suggests if it should tell the user a joke and play upbeat music. It can also ask "check-up" questions. The website also plots a graph of your emotions in a fixed time necessary for regular mental checkup.

  2. There is a friendly chatbot to answer your questions and can even alert the user's relative/friend through SMS if the user gets suicidal and depressed to a certain level. This can be of great help to save someone's precious life.

  3. We have a symptom checker section where you can just give your inputs step-by-step according to your issues and complaints to get a probable disease diagnosis and department of medicine to be contacted in the case of severity.

  4. Finally, we have a flowing UI catering to the needs of the users. We have tried to make the overall look and feel to make it calming, soothing, safe and engaging enough for our users.

  5. We also tried firebase authentication in google cloud systems and analytics.

How we built it

System Design

Video Emotion Detection

Emotion detection was also doing using Face recognition API. We used a custom open-source model that we found online to help us process user's emotions. We then extracted the user emotions from the API and connected them to timers on the frontend. This allowed us to have a Hashtable (using chart.js) that contains all of the data on how many minutes the user feels each emotion during the duration of the video recording.

Playlist Suggestion

We suggest playlists according to the mood of the user using Spotify API.

Chatbot alert system

We implemented a friendly chatbot using JavaScript and the alternative phone number of the user's relative stored in database was used by Twilio API to alert that the user is in danger and he/she needs help.

Symptom Checker System

We matched symptoms using Symptom Checker API. This was done in the backend using PHP and JavaScript.

Frontend

The frontend was built from a wide variety of web technologies. To start, we used JavaScript, HTML, SASS and CSS to form the core of our frontend stack. There were a lot of challenging components that went into building this project, and we made sure to take care of every single minor detail.

Backend

The backend was built using the PHP and MySQL framework PHP and JavaScript to make calls to various APIs and organize the data in a way that is presentable to the users.

Challenges we ran into

We were literally racing against time. The building of a good UI with all kind of functioning of different APIs and integrating each of them according to our user's need was indeed a big challenge for us. This was our first time utilizing Firebase to implement the login authentication of our website. We took the time to learn and understand how Firebase works and tried implementing it into our website. Unfortunately we were unable to deploy our web app successfully due to the problems in using multiple pages and php files.

Accomplishments that we're proud of

This project was especially an achievement for us because we created many additional components on the front-end in order to have a completed project. We are also particularly proud that we were able to include a vast array of different features for the user, which we felt makes BodyFrame such a serviceable form of digital healthcare. Lastly, we think the impact our project could have is a significant accomplishment. Especially in the digital age combined with remote day-to-day communications, this could really be a product that people find useful!

What we learned

We have a learnt a plenty much investing time and efforts in this project. Use of various APIs and integrating them plus a UI design to implement in front-end in such a short span of time was unbelievably challenging.

What's next for Body Frame

Checkup has so many useful features, so we'd love to further expand the project to handle direct 1:1 video calls with certified doctors! Overall, we hope that one day this project can be widely used as a practical and accurate resource for body and mental checkup.

Built With

Share this project:

Updates