Inspiration

Current healthcare systems are not prepared for a pandemic. So the questions are: How can we prepare for the next pandemic? How can we improve the current healthcare system to adapt?

Due to the highly infectious COVID-19 virus, many caregivers are unable to provide care or reach people who need care. The elderly and other care recipients are suffering from chronic diseases that require routine checkups. However this may not be as possible due to COVID-19 crisis and lockdown orders. Furthermore, unmanaged chronic diseases also increase the susceptibility of care recipients from suffering life-threatening complications from COVID-19 infection.

We envision that the future healthcare system should be both preventive and personalized. Our idea is to setup a platform to connect caregivers to elderly and care recipients that required care for chronic conditions, and monitor the health of elderly on a daily basis.

Our solution

Our solution is a two-way app platform for caregivers and seniors who required regular health check up. The system leverages AI technology to analyze data collected from facial recognition, speech recognition, wearable devices and/or IoT on a daily basis, and alert the caregivers if there is any identified risks. The platform also provides a way to facilitate communication between caregivers and care recipients, while aiding with health management to help alleviate caregiver stress.

Main features

Health data collection

  1. Facial recognition
  2. Speech recognition
  3. Chatbot
  4. Phone sensors
  5. Wearable devices/sensors

Elderly focus

  1. Voice control
  2. AI Chatbot to stimulate human interactions
  3. Enlarged text and other accessibility features
  4. Reminder system
  5. visual and sound alerts can be snoozed until the elderly login and complete the health monitoring daily

Data visualization for caregivers

  1. Data analytics dashboard
  2. Detailed health reports of elderly
  3. Alert system for identified issues
  4. Communication and reminders

How We built it

Software

This was an exciting project with many brains and skillset at work.

We preferred open source tools and platforms in different parts of the project.

  1. Frontend Dev using Angular, FireBase Authentication.
  2. Node Libraries Likes charts.js PWAs, BootStrap, Material Design, etc.
  3. Hosting and CICD setups using Netlify and Heroku and GitHub.
  4. Domain and SSL certificate from Namecheap and Let's Encrypt.
  5. SQL DB connected to the app with Restful API.
  6. Invision and Figma - UX/UI ProtoTypes and WireFrames
  7. Slack for Internal Communications & Google Drive for Documents, Images, etc.
  8. Google Colab notebooks to execute heavy GPU workloads and ML Algorithms.
  9. Slidego, Powtoon and Toonly for Video and Pitch Decks.

Machine learning

We collected datasets from varies sources such as Kaggle, JAFFE and IMFDB and trained the machine learning model for a couple of tasks: the identification of emotions from facial expressions, identification of BMI from face images, identification of emotions from speech, and detection of falls from phone sensors. Determination of cardiovascular disease risk is also achieved by reviewing cohort studies and results in medical journals. After training the model, we deployed a demo of the emotion prediction model, BMI prediction model, and cardiovascular disease risk using Heroku service.

Challenges we ran into

It is difficult to find quality labelled data for training machine learning models, which in turn affects the accuracy rate. Given that this is a remote hackathon, we were also unable to test connection with wearables. While there is flexibility to use the app without external sensors, we plan to integrate with multiple wearable devices and platforms in the future.

What's next for miia

We are planning to bring the project to the next stage. Shoot us a message if you're interested!

Built With

+ 4 more
Share this project:

Updates