Inspiration
Delivering a user-friendly, competitive, and easily accessible scribing tool, to ensure a quality standard amongst all clinics large or small. This tool will help with scribing, translating, and assisting doctors with diagnosing patients.
What it does
The current application takes advantage of federated learning to help train a model. Our app records conversations between a doctor and a patient and translate, summarizes, and transcribes them. This transcription is passed to Gemini which then polishes the summary. The doctor can edit and verify this summary. Based on the doctors' editorial feedback of this summary, a weighted score is given. This score is used to trained the model without exporting sensitive raw patient data to a server to maintain HIPAA compliance.
How we built it
We used concurrent prototyping with AI to build a tech stack. Pair-programming to flesh out features and bugs
Challenges we ran into
One of the main challenges we ran into was SDK compatibility during mobile app integration.
Accomplishments that we're proud of
- Creation of a successful federated model learning and workflow
- HIPAA compliant data processing (patient privacy is preserved)
- Mobile app integration
- Firebase server integration
What we learned
- Rapid prototype deployment
- Integration of federated learning
What's next for MedScribe
Our future goals are to fully train our developed federated model and outgrow reliance on Gemini for training the model.
Built With
- claude
- firebase
- gemini
- react-native
- whisper
Log in or sign up for Devpost to join the conversation.