Inspiration
The inspiration for DocNotes.ai stemmed from observing the significant administrative burden placed on healthcare professionals. Doctors and nurses often spend hours transcribing patient interactions and documenting medical charts, time that could be better spent providing direct patient care. We envisioned a solution that could automate this process, reducing burnout and improving the efficiency of medical documentation. Our goal was to harness the power of AI to create a tool that not only transcribes audio accurately but also structures it into a professional medical chart, ensuring privacy and compliance.
What it does
By leveraging advanced AI technologies, DocNotes.ai automatically transcribes audio recordings of patient sessions and converts them into structured, professionally formatted medical charts, ensuring compliance and accuracy while protecting patient privacy.
How we built it
For the Transcription module, we started by integrating OpenAI's Whisper model to handle the transcription of audio files. The audio was split into smaller chunks using the pydub library, ensuring efficient and accurate transcription. For the Redaction and Structuring Module, we used the Anthropic API to process the transcribed text, transforming it into a structured medical chart. This step also involved automatically redacting any personally identifiable information (PII) to maintain confidentiality. We developed a workflow to integrate these components seamlessly, ensuring that the final output was a clean, well-formatted medical chart. The output was then saved to a text file, ready for inclusion in patient records. Lastly, we designed a simple and intuitive interface where users can upload audio files and receive structured medical charts as output, making the tool accessible and user-friendly.
Challenges we ran into
Initially we started by trying to use the AWS Bedrock API and use the Claude models through those API calls, but we found that thorough documentation was a significant issue with Bedrock, disallowing us from finding the correct way to send system prompts to the model, which is a crucial component of the project. Additionally, for the transcription module, we also faced some issues in dividing the audio into smaller chunks and integrating the transcript into one, especially when our code was transferred between Windows and MacOS devices. Additionally, given the sensitivity of medical data, implementing robust methods for PII redaction and ensuring compliance with privacy regulations was crucial as well, which was a challenge we had to consider while building this project.
What we learned
This project gave us a deep insight into the world of speech-to-text transcription models, ensuring privacy and compliance with patient data. In addition, the importance of good prompt engineering was a significant lesson, given that the prompts that we provided and the smallest of change in details significantly affected the output.
What's next for DocNotes.ai
After this hackathon, we hope to further develop DocNotes.ai by ensuring thorough HIPAA-compliance and potential integration into the current medical system. We hope to support this service with a more polished web application, and
Log in or sign up for Devpost to join the conversation.