Eyes are the window to your soul, but fingernails are important indicators of your health. Based on observations of the fingernail, doctors can make oftentimes crucial health recommendations for further testing. While some abnormalities may be harmless, others, such as discoloration or thickening, can be early signs of serious health problems including liver and kidney diseases, heart and lung conditions, anemia, diabetes, and cancer.
What it does
Dexterity allows users to upload a picture of their fingernails. We use computer vision, made possible through a convolutional neural network, to automatically check for early signs of serious health problems.
How I built it
We scraped the web for images of healthy nails and images of nails that indicated some sort of disease, such as dark lines underneath the nail, white nails, yellowish nails, Beau's lines, or Terry's lines. We then trained a convolutional neural network on this data using transfer learning with the VGG ImageNet neural network. This allowed us to classify user-uploaded images into the five categories of nails listed above. Through techniques such as batchnorming, application of dropout layers, and image augmentation, we increased the accuracy of our classifier to ~86%, far past expectations for a dataset of the size we used for the purposes of this hackathon. This was made possible through use of a special-purpose AWS P2 instance with a Tesla GPU. Because our model data ended up being too large for GitHub, we used AWS S3 buckets to hold our data.
Challenges I ran into
We ran into trouble connecting the front-end and back-end of our product; as none of us had experience with servers, it proved to be very difficult to set up a public-facing server to access through Duke's WiFi network.
Accomplishments that I'm proud of
We are proud of how our product has the potential to encourage people to go see a doctor if they really need to. As people who observe abnormalities in their fingernails often refrain from visiting a doctor ("it's just my fingernails"), we hope our creation will help motivate people to contact a doctor early if they show problematic signs.
What I learned
We learned how to create a webpage from scratch and how to create a multi-class convolutional neural network.
What's next for Dexterity
We will want to get a larger dataset so that we can increase the accuracy of our model. The main bottlenecks to our current product is the lack of connection between the user-facing frontend and the neural network in the backend as well as the lack of data available to us in this short period.