Our interest in pursuing this project was sparked by the healthcare oriented start-ups that were present at the HackBeanpot (PathAI and Kyruus). We decided to delve into the most prominent issue at the forefront of healthcare - clinical diagnosis.

Recently, there has been a huge rise in the use of machine learning and artificial intelligence in medical research from improving breast cancer diagnosis to detecting diabetic retinopathy in older patients. Nevertheless, the inaccessibility of healthcare still stands in the way of those that are not able to afford the incredibly high charges that come with obtaining a diagnosis, much less treatment.

What it does

In order harness the rise of technology literacy across all sub-populations in the U.S. to provide a new route for healthcare, we built a web app that provides real-time diagnosis using a simple image or data upload.

By leveraging open-source datasets and the machine learning technique of transfer learning, we created a Minimum Viable Product of a web interface with an incredibly straightforward user experience.

Our web app takes in images from a person's cell phone and uses cutting-edge machine learning models to provide them with real time diagnosis for a given disease, capitalizing on the growth of cellular connectivity and innovations in artificial intelligence.

How we built it

We built this web app by using a node.js front end, equipped with image upload handling, command-line interaction to run the appropriate functions on the backend. The backend was developed using Python and two of the most popular Deep Learning frameworks (Keras and TensorFlow). The web app takes user input for the disease they would like to test for and uses a simple file upload to direct the ML algorithm to the correct path of the image.

The backend of the application process the image, reshaping and correctly setting the color channels for the image to match the structure of the input images/data that the architecture was trained on.

The ML algorithm then returns the probability of the diagnosis (confidence level) for the given disease.

Challenges we ran into

A major challenge we ran into was connecting user upload and input to the ML models so currently, this piece is not fully functional due to a lack of time.

Furthermore, we ran into issues with actually processing the images that we were manually feeding to the algorithms due to the intensive preprocessing that is done on training data but was not done by the user as these were raw, cell phone camera images. As a result, we had to replicate an image preprocessing pipeline for the images to ensure compatibility.

Unfortunately, we were not able to complete the process of fully integrating the front end upload with the backend execution but have them individually processing.

Accomplishments that we're proud of

We are proud of the fact that we designed an application that genuine has the potential to be implemented in the real world a preliminary connection for those unable to access healthcare today to be able to get an initial consultation through their cell phone camera. Not every application that can revolutionize the problems we face today has to build everything from scratch. By appreciating this principle of impactful simplicity, we are proud that we connected two industries that are growing rapidly (cellular connectivity and AI) to create a powerful product that can make a difference in peoples' lives.

What we learned

We learned the importance of the inter-connectivity of front end and back end development. We had to rapidly pivot our approach to the UI and UX based on who our application was aimed at (transitioned from hospitals to day to day use).

What's next for Diagnoser

We hope to smooth out our connection between front end and back end to ensure secure data transactions - currently we do not collect any account data to ensure no personally identifiable data is collected on our site. Furthermore, we would like to provide the option for people to be able to download their results and provide a private key to their healthcare provider to access their assessment and compare the diagnosis to their own judgement. Finally, we would like to create a stream-lined process for researchers to upload and connect their own models to the platform to make the app a globally open sourced project that can change the lives of people all over the world.

Built With

Share this project: