MelVision

Inspiration

One person loses their life to melanoma every 54 minutes. It is estimated that over 87000 people in the US will be diagnosed with melanoma this year and that over 9500 people will lose their lives this year due to this condition. This condition is diagnosable and curable in its earlier stages. However, this condition is often diagnosed in the later stages when it is incurable because of the specialized equipment and cost involved in diagnosing the disease. This is a problem which needs to be solved because of the potential to save a large number of lives with an early diagnostics test. Additionally, making such a service available to people in third world countries where such tools aren't available would enhance the quality of life of the people impacted by this service!

What it does

MelVision is a Melanoma diagnostic tool that can be accessed by anyone anywhere with a phone and a camera. It is a simple to use chatbot service accessible via SMS. You can start talking to the bot in real time by texting it and following the instructions from there. It is able to give you a live diagnosis of whether or not you have any stage of skin cancer in a few seconds after you send the image to the phone number of the service. The image is processed and then the results are sent back to the user along with the confidence in that image.

How I built it

This hack is primarily built on Microsoft technologies. The integrated service runs on Microsoft Azure while utilizing LUIS for understanding the context of the text messages, Custom Vision API for diagnostics and Computer Vision API for detection to verify the diagnostics. The hack also uses Twilio to communicate with the user via messages and the server via web hooks. Data for the skin cancer is available at ISIC Dataset which has over 13000 images.

Challenges I ran into

Getting everything to work together. Training the custom vision API. Building the SMS service. Having all the components run and interact in very specific ways took a lot of time.

Accomplishments that I'm proud of

Having a functioning service that is able to diagnose melanoma from anywhere in the world!

What I learned

How to use Azure, Custom Vision API, Server management, and Twilio in unison!

What's next?

Connect this to a patient portal and make some sort of message data transfer model.