UltraAssist

UltraAssist: An ultrasound assist for pregnant women (35.224.138.39)

Inspiration

In the US around 10% of the female population or 16.5 million females are pregnant. Out of which a third of them require regular assistance which includes regular checkups in the clinic, accounting for around 5.5 million women in a year or around 150 daily cases. But due to pandemic pregnant women are reluctant for in-person visits to the clinics.Also they are more sensitive to getting infected during in-person visits. With everything happening virtually these days, people are trying to utilize technology as much as possible. In this scenario, portable ultrasound equipment currently available in the market can also be used for at-home checkup. But as individuals with no medical background, it’s hard for them to navigate around the ultrasound, exactly to locate the probe at desired locations for accurate measurements and analyze the ultrasound images, especially when there isn’t a physician/nurse available for this assistance.

What it Does

We propose a physician guided virtual assistant for pregnant women to conduct ultrasound scans at home without assistance from medical personnel. The application will also show labels of organs to the user for better understanding of scans. There will be two login options, one for physicians and one for the user. Once the web app opens on both ends, the physician will point out the position for the probe to be placed; the patient will accordingly place the probe for efficient image capturing.

How we built it

The application was built as a responsive web-app (can be used both one mobile and desktops) using ReactJS as the frontend with Flask (python) and Google Cloud APIs as the backend. The app starts with a splash page, where the user has an option to either continue as a patient or a physician. The patient is shown an image of the abdominal area and the same image is shown on the patients end to create a common point of reference. The physician indicates which part of the abdomen needs to be scanned and it is shown to the patient. Once the patient has performed the scan of the region, they can upload and send it to the physician. At this point, the backend processes the ultrasound images using the SonoNet Neural Network model (on the Google Compute Engine) , classifies what can be observed in the ultrasound image and sends the classification and image to the physician. The physician can then either end the analysis or indicate the next area that needs to be analysed.

Challenges faced

Main challenge was to synchronize ultrasound probe positions at physician and user end.

Accomplishments

A web app was created using various tools which would really be helpful for pregnant women not only during the time of pandemic but also in normal situations.

What we learned

We learnt implementation of technical tools like ReactJS, Flask and GCP. We got exposure to SonoNet. It was a great experience to work virtually and complete prototyping of the project in such a short period of time.

What's next

There are a lot of potential features that can be integrated in the current prototype. Artificial intelligence algorithms can be used to detect any pregnancy related complications. Deep learning based image segmentation can be implemented for accurate labelling of organs. A chatbot can be developed for probe positioning during the ultrasound scan.