Inspiration 💡
For 4.5 years, I worked in the Emergency Department of a Level II Trauma Center. I witnessed firsthand how medical conditions, such as High Blood Pressure, caused Heart Failure when exacerbated and left untreated. For a while, I believed this was secondary to the patients, not taking their health serious. Truthfully, they presented to the hospital because they wanted to live, our healthcare system simply failed to present critical information to them in a way they could understand. In the USA, 100% of physicians are highly educated. Only 36.5% of Americans have at least a bachelors degree according to a study by the census in 2019. I noticed that the information seemed to get through to the patients when doctors would draw out information on paper or dry erase boards. Although they have the best intentions, physicians are often not artists and only have a certain amount of time to spend with a patient. A solution was needed to combat this disparity along with the impending physician shortage.
What it does 📝
DiagnosX is a Health/Medicine application that uses Google Gemini and 3D models to illustrate breadth and depth of a patient's diagnosis and medical history for thorough understanding. The interactive 3D models isolate the body system in question, for a full 365 degree view and marks critical regions pertinent to the medical condition. Patients can view the models of their diagnosis 24/7 and fully break down all aspects of their medical condition.
How we built it 🛠️
DiagnosX was built using Google Gemini and the Three.js library. For Functionality, DiagnosX can be successfully installed by:
- Downloading the DiagnosXAfroTech repository from Github
- Creating a .env file and Inserting Google Gemini API key as listed in the repository's README file
- Installing the dependencies with
npm ifollowed bynpm run startgemthen in a separate terminalnpm run dev
The application should then run on the local host of your computer for successful Platform execution.
DiagnosX is an Existing app, but the entire codebase submitted for this Hackathon is new and was written during the Hackathon Submission Period. Only the Logo and Idea are reused from the existing DiagnosX application.
The 3D models are the only Third Party Integrations for this submission, however, they are all free to use with proper CC attribution which is listed out in the Google Doc below.
Challenges we ran into 🚫
I got a really bad cold towards the end of the hackathon. Prioritizing my health for a bit delayed integrations of addition features I wanted this submission to have such as:
- Speech-to-Text
- Animation of 3D models
- Labeling of 3D models to pinpoint pertinent regions of models
Accomplishments that we're proud of 🏆
Despite the illness, I was able to get the barebones functionality of the application done. I am proud to have a functional prototype and getting an idea with the potential to impact millions via decreasing disparities in healthcare out there.
What we learned 🤯
Google Gemini is very easy to integrate and its language processing makes it great for interacting with users to answer questions. I did find information about customizing the Gemini responses to prevent hallucinations and/or to prevent the model from returning unrelated information to the patients but I ran out of time to integrate it with this submission. This will definitely be added to the existing DiagnosX application in a later update .
What's next for Diagnosx 🚀
- Adding text to speech for a more connected experience. The application reading out information makes it feel slightly more like engaging with the doctor.
- Getting models of every part of the human body.
- Add more animations to illustrate prognosis of medical condition as well as concerning symptoms.
- Include 3D models of drugs to illustrate how they interact with the body.
More information here: ⚜️
Built With
- google-gemini
- javascript
- three.js
Log in or sign up for Devpost to join the conversation.