Skin alarm

• 

  This is our prototype we apply to develop this application in the future with the following user dising

Inspiration

During the Covid19 pandemic a lot of outpatients with ongoing medical treatments were indirectly affected by the biosecurity measures proposed by the governments to stop the spread of the virus. In this context, non-urgent health matters such as ophthalmology, dentistry and dermatology were considered low priority and, in some cases, appointments were cancelled or postponed in an effort to avoid human face-to-face interaction as much as possible. In other cases, appointments turned into phone calls or video examinations making it hard for the practitioner to give accurate diagnosis.

We understand this can be frustrating based on our own experience, however, as a group of technology enthusiasts we believe in the power of machine learning and data-driven decisions to solve the challenges faced by any industry. For this hackathon, we have focused on the dermatological sector and we want to make skin-related distanced diagnosis a positive experience for both practitioners and patients.

What it does

We have designed and android app that embeds a CNN architecture deep learning model to classify nine different skin lesions. The app makes uses of the smartphone camera, so a photo of the skin-related problem is taken and sent to the neural network to predict to which type of skin lesion it belongs. For each skin condition, there will be a prescription and will tell if the patient is required to make a face-to-face appointment with the practitioner.

How we built it

To build our deep learning model we have used Python with the TensorFlow, TensorFlow Lite and Keras libraries. This CNN consists on three convolutional layers that extract the main characteristics of the images followed by two hidden layers (MLP) for image classification. Finally, an output layer with a SoftMax activation function that allows multiclass classification.

For our android app we have used Java in Android Studio. The app integrates the AI model (developed in python) and implements a prediction function that allows to capture the bytes of the image taken by the camera and tell a possible skin lesion.

Challenges we ran into

• Improve prediction accuracy as the dataset we worked with was relatively small (2241 images).
• Convert our Keras model into TensorFlow Lite (for android purposes) due to our lack of experience in machine learning for smartphones.
• It was the first hackathon for three of our members and one member is in a different time zone.
• Work with a multidisciplinary team (comp science, engineering, business and medicine)

Accomplishments that we are proud of

• We all improved our teamworking and communication skills.
• Our medicine student member became familiar with artificial intelligence in the health sector.
• Our business student member learnt how to build android apps and got UI design experience.

What we learnt

• We learnt how to use Tensorflow Lite to implement AI models in smartphones.
• We learnt about nine different types of skin lesions and their treatments

What's next for Skin Alarm app

• We would like to implement a system able to schedule face-to-face appointments based on the priority of the disease.
• We plan to implement a scheduler that advises when, at what time and in what dose a medicine should be taken.
• We aim to expand Skin Alarm to other types of diseases (e.g. chest x-rays to detect lung-related problems or x-rays to detect bone-related fractures)
• We plan to implement our app prototype design

Video: https://youtu.be/NW_5fn6e5Ms (the audio is broken)

Domain.com

https://ai-recommends-paracetamol.tech/

Built With

• android-studio
• keras
• python
• tensorflow
• tensorflowlite