Medscribe

Bridging the gap between patients and doctors

Comment

Inspiration

For our hack, Luis’s presentation about patient safety really inspired our group to take action towards the ever growing issue of medical errors. In his presentation, Luis mentioned that there are nearly 250,000 deaths in the US alone due to medical errors, that's almost 1 in 10 people admitted to the hospital. After doing some background research we found out that most of these deaths are due to medical miscommunication. To tackle this serious issue, we decided to create MedScribe, a transcribing device that records patient - doctor interactions and provides further medical insight. MedScribe will allow patients to easily access and review any important information discussed on a web app such as diagnosis, treatments, medication, dosage, etc. Throughout the creation process we faced many challenges and learned a great deal.

What it does

MedScribe is a transcribing device that records patient - doctor interactions and provides further medical insight. MedScribe will allow patients to easily access and review any important information discussed on a web app such as diagnosis, treatments, medication, dosage, etc. Throughout the creation process we faced many challenges and learned a great deal.

How we built it

We developed our project using OpenAI APIs and Google's Speech-to-Text API as the backbone. The front-end was created with React and personalized with custom CSS, with all the core functionalities handled through JavaScript. To provide real-time transcription, we incorporated a WebSocket React Hook which not only facilitated immediate transcription results but also negated the need for post-processing.

Challenges we ran into

One of our primary challenges was achieving precise audio transcription, initially from a .wav file and later in real time. This phase required us to explore and experiment with various APIs and libraries including Google Cloud APIs and OpenAI's Whisper. Additionally, we were committed to ensuring the security of confidential conversations, aiming to process all data in-house to prevent any potential leaks. Fine-tuning the post-transcription process, particularly in crafting prompts and managing responses, was also a significant hurdle, requiring careful planning and development.

Accomplishments that we're proud of

We take pride in our effective prompt engineering which has allowed us to offer patients a transparent window into their discussions with doctors, facilitating easy access to essential information. Our application successfully integrates a transcription hook with a patient-doctor classification system, offering an intelligent conversation assistant within a user-friendly interface. Additionally, we emphasized creating an accessible application, managing to score zero accessibility issues in the AXE accessibility test, and ensuring the app is usable by visually impaired individuals, embodying our belief that every patient deserves effective communication with their healthcare provider.

What we learned

This project has been a learning curve, especially in navigating the realm of real-time transcription and the application of AI in healthcare communications. We understood the vital importance of data security in safeguarding sensitive information and the complexity involved in crafting precise and effective prompts and responses. Furthermore, our journey taught us the significance of accessibility in technology, reinforcing our commitment to creating an inclusive platform where every patient can communicate effectively with their doctor.

What's next for Medscribe

In our next steps, we intend to bolster the security of our platform by integrating a custom Large Language Model (LLM). This will create a richer user experience by providing more in-depth insights and functionalities. Additionally, we plan to introduce authenticated user profiles to enable individuals to easily track and revisit their previous consultations and discussions. To enhance the utility of our application, we are also looking to implement a feature that allows users to translate their conversations with healthcare professionals into different languages. This initiative is in line with our goal to facilitate clear and inclusive communication between patients and doctors, thereby elevating the standard of healthcare dialogue.

Built With

Share this project:

Updates