SageScript AI

Inspiration

Healthcare providers face two critical challenges: extensive documentation time and the complexity of evaluating patient histories effectively. Doctors often spend hours reviewing scattered medical records while simultaneously trying to document current consultations. This fragmented process not only reduces time with patients but also increases the risk of missing important historical context that could influence diagnosis and treatment decisions. SageScript AI emerged as a solution to unify these workflows - enabling real-time documentation while surfacing relevant patient history at the point of care.

What it does

SageScript AI serves as an intelligent medical assistant that:

• Transcribes medical consultations in real-time or alternatively from uploaded audio files
  
• Retrieves relevant patient history (allergies, medications, conditions) during consultation
  
• Generates structured clinical reports based on the transcribed consultation and patient history using an LLM
  
• Enables physicians to review and edit generated documentation
  
• Integrates with FHIR-based health records for seamless data flow
  

How we built it

While we used Groq APIs for speech to text and LLM functionalities, both models are small and lightweight enough for local deployment and real-time inference.

• Frontend: Streamlit
• Backend: MeldRx FHIR-compatible API
  
• AI Backend:
  
  • Speech to text: Groq's Whisper Large Turbo model to transcribe medical consultation audio files
    
  • LLM: Groq's Gemma-9B-it for context-aware medical report generation
    

Challenges we ran into

Understanding what FHIR is, building the app was easier.

Accomplishments that we're proud of

• Created a functional prototype with real-time transcription and patient context aware report generation.
• We talked with medical practitioners to understand what they need help with and built our app based on existing problems and complaints.

What we learned

• Some problems are critical but unsolved mostly due to the high entry barrier to the medical domain and issues around data privacy.
  

What's next for SageScript AI

• Better filtering of diagnostic reports and observations
  
• Adding medical imaging results as context
  
• Multi-language capabilities, both Whisper Turbo and Gemma are multilingual and yielded great results when tested with mock consultation recordings in German and Turkish
  

Built With

• gemma
• groq
• meldrx
• python
• streamlit
• whisper