ConvMed - Healthcare Social Media Platform For Hospitals

An AI powered social media mobile application for hospital staffs/doctors to upload, analyze, save,download electronic records and chest X-Rays seamlessly powered by AWS S3, Lambda, Comprehend Medical

Comment

Inspiration

Social medias like instagram, facebook and whatsapp has become our day to day life and we are even so attached to the user experience of these social media applications. We thought what if we can develop a social media application for hospitals which can be used exclusively to analyse-share-save health records, chest X-Rays and medical records.

What it does

ConvMed is a social media mobile application for hospitals to analyze Electronic Health Records, medical tests, Chest X-Rays, etc powered by AWS Comprehend, AWS Lambda and AWS EC2. The analysed results are stored as comments in the post and can be accessed by doctors, lab assistants any other healthcare workers within the hospitals protecting the privacy and security. The documents/images used in the applications are sample.

How we built it

Architecture

Alt text

The app features

  1. Analyse medical test reports/EHR using AWS comprehend

  2. Analyse chest X-Ray using Torchtorchxrayvision library with AWS EC2 C5.Xlarge instant backend

  3. Signup, login, profile picture-status updating, follow, unfollow, like count, saving, downloading the posts across users

Usage Of AWS Comprehend Medical And Other AWS Services

Document Analysis

When user upload an image without switch toggle for chest x-ray detection and click on analyze, the image gets uploaded to AWS S3 using cognito pool id in kotlin, with the filename as firebase uid. Then a lambda function will be triggered using API Gateway using the image filename and the image passed through AWS Textract and the returned text string used by AWS Comprehend with result indents are returned as response for the mobile application. The user details are added in the AWS dynamodb database

./lambdafunctions/comprehendfunc.py

Chest X-Ray

We deployed flask application on AWS EC2 instance using Gunicorn and nginx with torchxrayvision library support. TorchXRayVision is an open source software library for working with chest X-ray datasets and deep learning models. It provides a common interface and common pre-processing chain for a wide set of publicly available chest X-ray datasets including famous datasets such as MIMIC-CXR (MIT) and NIH chest X-ray8. In addition, a number of classification and representation learning models with different architectures, trained on different data combinations, are available through the library to serve as baselines or feature extractors. When user select and click on analyze button with toggle switch on the image uploaded to aws s3 using cognito pool id in kotlin, with the filename as firebase uid. Then a lambda function will be triggered using API Gateway with AWS EC2 instance url(flask app) along with parameter.

./lambdafunctions/Chestxray.py

It will return 18 classes

Atelectasis

Cardiomegaly

Consolidation

Edema

Effusion

Emphysema

Enlarged_Cardiomediastinum

Fibrosis

Fracture

Hernia

Infiltration

Lung_Lesion

Lung_Opacity

Mass

Nodule

Pleural_Thickening

Pneumonia

Pneumothorax

App is Built With 🛠

  • Kotlin - First class and official programming language for Android development.
  • Coroutines - For asynchronous and more..
  • Flow - A cold asynchronous data stream that sequentially emits values and completes normally or with an exception.
  • Android Architecture Components - Collection of libraries that help you design robust, testable, and maintainable apps.
    • LiveData - Data objects that notify views when the underlying database changes.
    • ViewModel - Stores UI-related data that isn't destroyed on UI changes.
    • ViewBinding - Generates a binding class for each XML layout file present in that module and allows you to more easily write code that interacts with views.
    • DataBinding - Binds data directly into XML layouts
  • Dependency Injection -
    • Hilt-Dagger - Standard library to incorporate Dagger dependency injection into an Android application.
    • Hilt-ViewModel - DI for injecting ViewModel.
  • Backend - Google Firebase
    • Firebase Auth - To support email based authentication
    • AWS DynamoDB - A NoSQL database to store all data
    • AWS S3 - A cloud storage to store all images
    • AWS Comprehend - A cloud service to analyse medical text
    • AWS Textract - Document to text OCR cloud service
    • AWS EC2- Cloud based virtual computers
  • Retrofit - A type-safe HTTP client for Android and Java.
  • GSON - A modern JSON library for Kotlin and Java.
  • Timber - A simple logging library for android.
  • GSON Converter - A Converter which uses Moshi for serialization to and from JSON.
  • Glide - An image loading library for Android backed by Kotlin Coroutines.
  • Material Components for Android - Modular and customizable Material Design UI components for Android.

Challenges we ran into

Building the android application with multiple features and analysis, making the restful application, lambda function, s3 managment.

Accomplishments that we're proud of

Building a fully fledged applications using the power of AWS Comprehend Medical

What we learned

Building sophisticated mobile application with bundle of features from AWS, Kotlin and AI in one place.

What's next for ConvMed - Healthcare Social Media Platform For Hospitals

Download the results as CSV files, making the returned results more attractive by elimintating the dictionary type responses.

Share this project:

Updates