When it comes to your health, timing is critical and all people deserve to get the best medical advice there is. What we wanted to do is come up with a way to create an artificial intelligence that learns how to make medical diagnoses by analyzing large amounts of medical images. The larger the data-set, the more accurate it could be. A successful system would be able to make a correct diagnosis more quickly than a single doctor and possibly more accurately.

What it does

DiagnosPics is a website where doctors are able to make a medical diagnosis by uploading medical images. Currently it analyzes two different image sets: x-rays of the chest and ultrasounds for breast cancer screenings. The system compares the uploaded image to the machine learning model and determines which of the trained outcomes it is most similar to. The image and prediction are displayed on the screen.

How I built it

The DiagnosPics website was built using the Wix web development platform. Because using vision with machine learning is still pretty new, we built the models with two different vision platforms. The breast cancer screening model was built using Microsoft Azure's Custom Vision API. The chest x-ray model was built using Google's AutoML Vision. Both were built using the process of tracking down officially labeled open-source data-sets of the images, categorizing them for the model, then training the model with the images. After seeing the Azure was more compatible with the Wix platform, we created a version of the x-ray model with Azure for the website. The accuracy of both models was quite high for how long we had to develop them.

Challenges I ran into

Connecting both APIs to the Wix Code framework was not easy. Working with software tools still in beta, meaning there was little troubleshooting support.

Accomplishments that I'm proud of

Not building just one, but two fully-fledged vision learning models using cutting-edge software. Working with incredible people to build a working piece of tech.

What I learned

Machine learning models take a lot of testing. Not all data sources are created equal. Connect the APIs earlier in the process.

What's next for DiagnosPics

This is large space and there is a lot of room for expansion. For one thing, there is a need for a larger amount of imaging data to create a more robust ML model. Moreover, this technology can be further grown to study not only 2D but also 3D images, and become a diagnostic tool that could almost replace differential diagnosis, thus eliminating human error in medical diagnosis, while also saving time.

Built With

Share this project: