Patch

Patch is healthcare's versatile virtual assistant, a streamlined platform for those in need of quick & easy medical guidance and information, derived from a single phone call.

Comment

Inspiration

I was inspired to do this because I have never done a project related to healthcare before, and forwarding the combination between software engineering and healthcare is what I am working towards in my life.

What it does and How I built it

Patch is a platform built from a Twilio integrated Flask backend that serves as the API for a React frontend. I leveraged Google Cloud Platform to train a deep learning model to classify sentences into multiple categories based on physical and mental symptoms. The user can call the Patch hotline and describe their symptoms and their medical concerns, such as if they still need their seasonal flu vaccine or a blood pressure check. The audio is recorded, transcribed, and discarded, and with some help of the natural language toolkit, the transcription is simplified into several sentences and fed into the neural network. An overall score for the entire transcription is calculated, as well searching for specific words and sentences of their medical concerns. A brief summary of the findings are described via messages, and the user is redirected to Patch, where they can read a personalized report that contains medical guidance tailored to their input. All reports are stored in a Datastax Astra database using CQL. The frontend features a mapbox instance populated by facilities that can perform vaccinations for the flu/coronavirus, tests for HIV and blood pressure, etc. which were geocoded using radar.io from datasets provided by the city of New York. The users' personal reports have their location plotted on the map, making it easy to see the closest of locations, and every point on the map has several viewable details, from address and phone number to prices and whether appointments are preferred over walk-ins.

Challenges I ran into

There were several challenges I ran into. The most notable is that the deep learning model overfit the data incredibly, so it definitely misunderstands sentences and often assumes them to be neutral, which is not helpful. I tried lots of things, including using stemmers from the Python natural language toolkit, but to no avail. I also really forced myself out of my comfort zone by purposefully using technologies I have never used before. But of course, the fun part is that uncertainty.

Accomplishments that I'm proud of

For the first time in my life, I made something I feel like other people could really get something out of. Most of the time I work on personal projects that nobody would have any interest in, but this is different. Sure, it's kind of hacked together in some places... okay, a decent amount... but I say it's just the beginning! I'm incredibly happy with what I've created and I'm thrilled to work on this, or the next best thing, further down the road. Oh, and this is my first hackathon, so thanks for letting me attend!

What I learned

Artificial intelligence is very defiant. It will trick you into thinking it's better than it really is. The dataset can't be problematic and highly repetitive like the one I generated to train it, or else it will overfit, which is bad. I'm excited to see how I can improve it now that I'm not under the time constraint.

What's next for Patch

My focus is definitely to improve the deep learning model. The next step would be a feature that allows users to organize the nearby locations by certain categories, like price or distance. I do think this idea could potentially go a long way to help people, so I really don't want to give up on it.

Share this project:

Updates