Over the course of Quarantine and the global pandemic we all are facing right now, I personally have started to think more and more about my life and my health in general. Oftentimes, we might feel that we have a certain condition when in reality it is only our fear getting the best of us. But how does one truly know whether the symptoms they are experiencing are to the actual extent they seem to be. As a result, in order to not only keep our mental balance in regards to paranoia of our own health, but also help keep front line workers addressing the major pandemic that has plagues the world we have create Skin Apprehensiveness Valdiator and Educator App, which allows users to detect, get educated about and get consultation regarding skin diseases using react-native, google-cloud and ML/AI model using tensorflow. Skin Cancer is dangerous --> 2 people every hour die from skin cancer, and 1 in 5 Americans suffer from skin cancer by the age of 70.
What It Does
The application consists of four main sections:
The main part of the app is the home page. It consists with a user-friendly UI that allows the user to decide whether they want to pick a pre-existing photo or take a current photo of any skin related concerns they may have. They are then able to choose or take a picture and then, upon submission, using google-cloud ML Kit, we are able to use the cloud in order to our model and receive results from the model. We receive our confidence level and our overall diagnosis of what disease our Model thinks it is. Following this, we also have the option to view more details regarding this disease through a wikipedia link or submitting this to a specialist to double check the diagnosis. The orginial Machine Learning Algorithm was made using tensorflow and a convolutional neural network with multiple hidden layers. It has a relu activation layer, and it uses the Adam optimizer. Along with this we have increased the accuracy with cross validation.
The second part of the app is the specialist section where you are able to view doctors who you can verify online regarding your diagnosis. You would be able to shoot them a text, email or leave a missed call to receive a consultation and discuss more regarding your diagnosis. This includes questions such as what should my treatment be, when should I start, where should I go, how much will it cost and other questions that pertain to your consultation.
The third part of the app is the practices section where you are able to find dermatology practices near you that are able to help you with your treatment. You are able to view a variety of locations including their Google Review Average rating, total amount of reviews, phone number, address and other such details. You also get a small insight into how their office looks. You are also able to click on the location and it will redirect you to the directions to that location using a Google Maps Link.
The final part of the app is the tips section where you are able to find various links from credible sources that will help you get advice regarding treatment, diagnosis or skin disorders in a general sense from various articles. This includes articles pertaining to various skin cancers, various skin disorders, chronic skin problems and what to do if you happen to contract a skin infection.
How We Built It
The technologies we used include
NativeBase.IO Library for certain UI Components
React Navigation Library for creating a comprehensive navigation scheme
Expo workflow for efficient testing and production
Google Cloud ML Kit for hosting our custom TF Lite Model on firebase making it faster and easier to classify models
Practo API Providing different Doctor and Practitioner Information to the application based on location, for the user to easily find
Google Places API for finding the closest practitioner locations nearby
Used Tensorflow and CNN's with hidden layers and an optimizer to detect skin cancer.
Challenges We Ran Into
We encountered a lot of issues while making the machine learning part.
- Finding a reputable dataset with training and test features was hard to find, especially as they didnt have enough images and had a low accuracy as a result
- Since we used an app, it was tough to convert the tensorflow code to tensorflow lite files
- As We are not too experienced in deep learning and machine learning, it was tough to learn all the concepts in such less time.
Difficulties Integrating New Apis due to incorrect object references
What We Learned
In the past we had used ML Kits pre-trained models but we hadn't used our own custom trained models before and as a result it was a completely new experience.
We learned how to use CNN's, and deploy tensorflow models.