Inspiration
According to Eksposure, a person takes more than 450 selfies a year and almost 100% of people care about the health of their face. So, our motivation is simple: bringing healthcare for facial health closer to us using technology. When someone have a mirror in front of them and see if their eyes, mouth, teeth, or face has any issues, they will run into the doctors. What if we can take selfie of ourselves and ask with our smartphones for example to examine us and even recommend medications and available doctors whom we can talk to? That's where we bring in the idea of dr. MARTHA.
What it does
Dr. MARTHA uses Computer Vision to identify key health issues and problems from the patient's photo of eyes, teeth, tongue, and face. It also recommends the treatment, medical procedure, and suggest professional doctors for further diagnosis. Users will interact with Doctor Martha from the application.
How we built it
We have developed a Flask application that allows to upload photos of their eyes, mouth, teeth, and facial skin and run computer vision model to identify any health issues from the photo. This application is run fully in Python. We have developed 4 separate YOLOv8 object detection model that has been trained on collection of photos with different classes of diseases and issues. The collection of photos for training is intricate.
We scraped pictures from the website and performed the labeling and annotation using Roboflow. For the UI of doctor Martha, we use HTML and CSS styles. Also, we simulated a database of medications specific to each diseases and also database of doctors with each professional information, in CSV. We also scraped the pictures of medications from the websites.
The following is the Architecture of our solution
Challenges we ran into
Detection accuracy is one of the most important challenge in the development of computer vision models because medical diagnosis need to be accurate. We also would like to integrate with database such as Mayo Clinic however our technique was still not effective. It needs to explore the best way how to integrate more information into our application.
Accomplishments that we're proud of
We're proud to develop this app because this technology can help a lot of people. Using this technology, the individual awareness on health will improve and easier accessibility to medical information. With this technology, we merged multiple health issues category into one personal application.
What we learned
While developing this solution, we learned that medical information is complicated and need high accuracy to find the right information which the application will give to the patient. It needs thorough research on medications and treatments. We also learned that technology is never 100% accurate. Therefore, the future roadmap of this technology needs collaboration with medical professionals.
What's next for Doctor Martha
Developing mobile app version with flutter, developing ML model for symptoms disease prediction for example connecting with database from Mayo Clinic, collaboration with doctors and medical professionals to test the accuracy of our model, embed with location tagging to match with nearby doctor database, and connecting to commercial drug stores to help people find medical help.
Log in or sign up for Devpost to join the conversation.