Here are some quick links to some of the resources we developed while creating our project:
Describe the problem your solution addresses?
As our population ages we will begin to have a lot of multimorbidities. The aging population will have higher rates of diabetes, hypertension, and other chronic ailments. Mobile health (mHealth) platforms using smartphones have proven effective for monitoring blood pressure, glucose and other health related symptoms. However, applications are not always accessible for the elderly population. Finger sensitivity and mobility can be an obstacle for the elderly as it impairs their ability to interact with apps. Features such as larger font size, high contrast, and text to speech functionality are often neglected due to the lieu of modern design trends intended to appeal to younger audiences.
Why did you pick this solution and how does it address the problem?
We designed our app, miia (Medical Intelligently Applied) to be accessible and usable by most seniors. Miia is an application that will help track and manage health conditions for the elderly population. For instance, we implemented a Chatbot function to help seniors input their vital signs. The chatbot can be made to speak aloud, while the senior can utilize their voice which is then converted to text. The chatbot can also ask questions to monitor symptoms and mood to screen for infection or depression, respectively. Furthermore, our app will track mobility and activity functions of our users through drawing data from the built-in accelerometer, gyroscope, and other smartphone sensors. This will help us predict activity level and potentially prevent frailty and traumatic falls with seniors.
How to use miia
Miia can be used through entering https://miia.me/ and signing in with gmail or by creating a new account. Once you've logged into miia you're greeted by the main dashboard that provides an overview of your profile along with several different tabs. Here users can chat with miia, sync wearables, and receive diagnostic reports from health checkups. Current functionality of the application is limited to conducting conversations with the chatbot while also completing facial recognition scans that detect mood and BMI.
Nonetheless, our current figma prototype serves as a better representation of the apps final functionality and design. In contrast to the web application the prototype is developed for mobile devices to better serve the elderly through prioritizing convenience and mobility. The prototype itself is fully interactive as users have the ability to click, scroll and drag through both caregiver and patient interfaces.
What it does
The system leverages AI technology to analyze data collected from facial recognition, speech recognition, wearable devices and/or IoT on a daily basis, and alert the caregivers if there is any identified risks. The platform also provides a way to facilitate communication between caregivers and care recipients, while aiding with health management to alleviate caregiver stress.
Health data collection
We ensure the health data collection process is easy to follow by having the whole health check up process guided by our AI chatbot miia, which include the following:
Facial recognition - facial image taken for analysis of cardiovascular diseases risks, emotions, BMI and etc.
Speech recognition - speech recorded and analyzed for emotions and mood
AI chatbot - collect health data unavailable in facial and speech recognition/ wearable devices
Phone sensors - detection of fall
Wearable devices/sensors - measurements including but not limited to blood pressure/ heart rate/ sleeping pattern/ activity
Elderly focus design
Voice control - elderly users can choose to interact with chatbot by voice or text
AI Chatbot to stimulate human interactions
Enlarged text and other accessibility features
Reminder system - visual and sound alerts can be snoozed until the elderly login and complete the health monitoring daily
Data visualization for caregivers
Data analytics dashboard - show key metric of elderly over one month
Detailed health reports of elderly - details of each health parameter
Alert system for identified issues - caregivers can set threshold values according to elderly's condition; red warning symbols and notification pop up when value above/ below normal
How we built it
• Frontend Dev using Angular, FireBase Authentication.
• Node Libraries Likes charts.js PWAs, BootStrap, Material Design, etc.
• Hosting and CICD setups using Netlify and Heroku and GitHub.
• Domain and SSL certificate from Namecheap and Let's Encrypt.
• SQL DB connected to the app with Restful API.
• Google Colab notebooks to execute heavy GPU workloads and ML Algorithms.
• Invision for developing WireFrames
• Figma for creating final prototype
• Slack for Internal Communications & Google Drive for Documents, Images, etc.
We collected datasets from varies sources such as Kaggle, JAFFE and IMFDB and trained the machine learning model for a couple of tasks: the identification of emotions from facial expressions, identification of BMI from face images, identification of emotions from speech, and detection of falls from phone sensors. Determination of cardiovascular disease risk is also achieved by reviewing cohort studies and results in medical journals. After training the model, we deployed a demo of the emotion prediction model, BMI prediction model, and cardiovascular disease risk using Heroku service.
Challenges we ran into
It is difficult to find quality labelled data for training machine learning models, which in turn affects the accuracy rate. Given that this is a remote hackathon, we were also unable to test connection with wearables. While there is flexibility to use the app without external sensors, we plan to integrate with multiple wearable devices and platforms in the future.
To facilitate the adoption of our technology, we plan to target caregivers (B2B) as our primary target demographic. Currently there are 34 million caregivers for the elderly in the United States, with 5 million of them being long distance caregivers. Our goal is to introduce our product, while increasing our adoption rate, and thus solidify our application as an essential tool for caregivers worldwide.
Currently miias distributions channels will be limited to mobile app stores found on both android and ios devices. In later iterations miia will transition to being available as a web application for desktops.
Our go-to-market strategy during distribution will include a combination of freemium and viral approaches. This in-turn provides us with financial incentives for early adopters, who are able to take advantage of the 2-month free trial while having the ability to subscribe later. We’d also like to introduce a referral system where users are able to promote our application while being rewarded for successful signups. In addition to this, we aim to partner with health organizations (clinics/ hospital/ national health insurance) alongside deploying through-the-line marketing tactics in order to enhance customer reach and maximize customer acquisition.
What's next for miia!
We are planning to bring the project to the next stage. Shoot us a message if you're interested!