miia | Devpost

Inspiration

As our population ages we will begin to have a lot of multimorbidities. The aging population will have higher rates of diabetes, hypertension, and other chronic ailments. Mobile health (mHealth) platforms using smartphones have proven effective for monitoring blood pressure, glucose and other health related symptoms. However, applications are not always accessible for the elderly population. Finger sensitivity and mobility can be an obstacle for the elderly as it impairs their ability to interact with apps. Features such as larger font size, high contrast, and text to speech functionality are often neglected due to the lieu of modern design trends intended to appeal to younger audiences.

Therefore, we designed our app, miia (Medical Intelligently Applied) to be accessible and usable by most seniors. Miia is an application that will help track and manage health conditions for the elderly population. For instance, we implemented a Chatbot function to help seniors input their vital signs. The chatbot can be made to speak aloud, while the senior can utilize their voice which is then converted to text. The chatbot can also ask questions to monitor symptoms and mood to screen for infection or depression, respectively. Furthermore, our app will track mobility and activity functions of our users through drawing data from the built-in accelerometer, gyroscope, and other smartphone sensors. This will help us predict activity level and potentially prevent frailty and traumatic falls with seniors.

What it does

The system leverages AI technology to analyze data collected from facial recognition, speech recognition, wearable devices and/or IoT on a daily basis, and alert the caregivers if there is any identified risks. The platform also provides a way to facilitate communication between caregivers and care recipients, while aiding with health management to alleviate caregiver stress.

How I built it

Software

• Frontend Dev using Angular, FireBase Authentication.

• Node Libraries Likes charts.js PWAs, BootStrap, Material Design, etc.

• Hosting and CICD setups using Netlify and Heroku and GitHub.

• Domain and SSL certificate from Namecheap and Let's Encrypt.

• SQL DB connected to the app with Restful API.

• Google Colab notebooks to execute heavy GPU workloads and ML Algorithms.

• Invision for developing WireFrames

• Figma for creating final prototype

• Slack for Internal Communications & Google Drive for Documents, Images, etc.

Machine learning

We collected datasets from varies sources such as Kaggle, JAFFE and IMFDB and trained the machine learning model for a couple of tasks: the identification of emotions from facial expressions, identification of BMI from face images, identification of emotions from speech, and detection of falls from phone sensors. Determination of cardiovascular disease risk is also achieved by reviewing cohort studies and results in medical journals. After training the model, we deployed a demo of the emotion prediction model, BMI prediction model, and cardiovascular disease risk using Heroku service.

Challenges I ran into

It is difficult to find quality labelled data for training machine learning models, which in turn affects the accuracy rate. Given that this is a remote hackathon, we were also unable to test connection with wearables. While there is flexibility to use the app without external sensors, we plan to integrate with multiple wearable devices and platforms in the future.