Due to the outbreak of COVID-19, medical consultations for non-urgent conditions are switched to online. Current telemedicine apps only provide video consultation and electronic medical records. Limited health information can be collected and the diagnosis process is ineffective.
Our idea is to increase the amount of useful information collected to aid diagnosis including facial recognition, voice recognition, and vital signs.
What it does
Patients can sign up with their email, name, and create a password
Patients can log in with their email and password
Patients can upload their photos to the system
Patients can get the results link from the system and forward to their doctor
Doctors will be notified by SMS when their patients upload a picture & report link is generated
Doctors can get the photo patients uploaded and the analysis by texting usernames
Eye analysis (crossed eyes, glycemia, cataract etc.) and water retention diagnosis
How we built it
Low fidelity wireframe using Balsamiq
High fidelity wireframe using Figma
front-end development with vue and element UI
back-end development with flask
features to send images and analysis after texting username using Twilio API
feature to notify the doctor by SMS when their patients upload a picture & report link is generated
Google could vision API for facial recognition
Trained machine learning model with keras
Challenges we ran into
We have limited datasets to train the model, thus the accuracy rate can be further improved with more labeled datasets. Also, cloud vision API does not support uploading only eye images (we have to upload image of the whole face), so we need to experiment with HaarCascadeClassfier from OpenCV.
Accomplishments that we're proud of
We have completed the UI design of the whole app, trained machine learning models to detect eye diseases and water retention, and the working prototype can be accessed via here
What we learned
We learn how to use Twilio to automatically send message and reply to a text from user, element UI and vue for front-end development.
What's next for Prognomic
- Patients can upload images of other parts of the body
Doctors can sign up with their work email, name, and create a password
Doctors can log in with their email and password
Doctors can import patient record or add patients during onboarding
Doctors can request photos from patients to upload to the system
Doctors can view the photos in the system and the machine learning model results
Doctors can add comments/ medications/ recommendations/ medical conditions to patient’s record