Checkup Health [Virtual]

Your Personal Psychopatholigical Health Assistant

Comment

Inspiration

In today's digital world affected by Covid-19, telemedicine is more important than ever before. With limitations on beds in hospitals, most people worry whether their symptoms are severe enough to require a doctor's appointment. Most cases are not, yet this worry leads to influxes in doctors visits which only prevents the real patients from getting their medical treatment on time.

What it does

Checkup is a bleeding-edge machine learning service that brings a doctor's appointment to the comfort of a user's bedroom. It allows regular people to assess their overall psychological states and mental well-being without the need to visit a doctor.

Using two metrics -- audio and video, Checkup analyzes a person's health and provides the following:

  1. Video Facial Recognition
  2. Video Emotion Detection
  3. Audio Sentiment Analysis
  4. Symptom Matching

Finally at the end, we generate a comprehensive (downloadable) report that one may choose to take to a doctor for further patient diagnoses.

How we built it

We built this entire project in TypeScript, Javascript, and CSS. We used Face API to perform emotion detection and video facial recognition, and Web Speech APIs to handle audio sentiment analysis. Finally, for Symptom Matching, we relied on the ApiMedic's Symptom Checker API

Challenges we ran into

Building this project was an ambitious feat for me just because this is one of the first times I've worked on Javascript/TypeScript-related projects. I had two teammates who were originally on my team, but they quit midway, so I was left to build the rest of the project. This warranted me pulling an all-nighter, but every moment was worth it!

What's next for Checkup

I plan on taking this project further. I hope to investigate other methods that can more accurately help a user assess their psychological well-being. Plus, there are a lot of external factors (like lighting) that can skew predictions one way or another. My plan is to address these issues and then publish this app to a wider market where I hope more people (students and working professionals alike) will benefit from this.

Built With

Share this project:

Updates