Inspiration
The journey of Docsy began with a simple yet profound realization: the heart of healthcare lies in communication. Doctors and patients engage in countless conversations, each rich with information that could significantly streamline patient care and documentation. We saw an opportunity to capture this invaluable data and automate the cumbersome form-filling process. The idea was to create a system that not only saves time for healthcare professionals but also enhances the accuracy of patient records—thus, Docsy was born.
What it does
Docsy automates the form-filling process in healthcare settings. It records conversations between doctors and patients and uses voice recognition technology to transcribe and diarize these interactions. Then, leveraging a Large Language Model (LLM), Docsy extracts pertinent Q&A pairs from the conversation, populating the patient’s forms directly in the system for review and confirmation by healthcare providers.
How we built it
We developed Docsy using a React-based frontend to manage the user interface and audio recording functionalities. The backend, powered by Django, handles the audio data, which is then processed through Azure AI's Speech API for transcription and speaker diarization. Post transcription, an LLM is employed to interpret and extract structured data (Q&A pairs) from the text.
Challenges we ran into
One of the main challenges was ensuring the audio recordings were in the correct format for Azure AI's processing needs. Additionally, integrating and effectively utilizing the diarization capabilities of Azure AI proved to be a complex task. We spent considerable time fine-tuning these elements to ensure reliable operation.
Accomplishments that we're proud of
We are particularly proud of developing a functional system that directly addresses a real-world issue—reducing the administrative overhead for doctors. Docsy's ability to accurately extract and populate data into forms has potential to significantly improve operational efficiency in healthcare.
What we learned
The project deepened our understanding of voice processing technologies and their application in real-world scenarios. We also gained insights into the complexities of healthcare data compliance and the importance of designing intuitive user interfaces for medical professionals. We also learned a lot in fine-tuning LLMs.
What's next for Docsy
Looking forward, we aim to enhance Docsy by adding more robust security features, including login functionalities to ensure data privacy and compliance with laws regarding patient information. We also plan to expand its capabilities to integrate seamlessly with other hospital systems and to offer customization options for the questionnaire to accommodate different healthcare providers' needs.
Log in or sign up for Devpost to join the conversation.