There is currently a need for remote access to precision forms of medicine and diagnosis. Finding a specialist or booking an appointment can be quite the challenge, especially during tough times like these. Due to these factors, a platform that can provide reliable, quick, and confident diagnosis is essential.

Patients in third world countries may need to wait months at a time just for triage [1].

What it does

ISight is a platform that aims to provide quick medical recommendations given a photo of a skin lesion. A user may upload a photo that is quickly sent off to our remote server for processing! ISight then provides a potential diagnosis along with a statement of confidence, just to make sure that you can get the help you need. Ideally, ISight should be used as a screening application to further investigate some suspicions you may have had.

How I built it

ISight was built as an android application and is hosted on a flask python server. A fleet of powerful nerual networks have been trained and ensembled to ensure you get the most bang for your buck! With the additional benefits of being able to estimate uncertainty along the way.

Challenges I ran into

Building an application isn't always easy! Managing large models on a small application proved to be harder than expected. Moreover, figuring out how to leverage an ensemble of neural nets (resnet18) and calibrating them to leverage confidence was a major challenge!

Accomplishments that I'm proud of

At the end of the day, we the team, are quite proud that we made it all the way through till the end. From frontend to backend, each and every one of us tackled new and exciting problems that we pushed through

What's next for iSight

Future plans for our application include improving the precision of our probability values, and even providing probability and confidence values for multiple diseases, which could help facilitate further diagnosis. In addition, we would intend on improving the backend by implementing a user interface for doctors where they can monitor and manage triage requests from a given uses as well as provide direct feedback that gets pushed directly to the users phone application. And of course, encrypting all this sensitive data properly.


Share this project: