UF Health SmartScribe

I haven't seen a doctor in 6-7 years, because eating an apple daily is faster than a checkup. But now, with UF Health SmartScribe, one chat with my Doctor, and the only thing left to do are my shots.

Comment

UF Health SmartScribe

Inspiration

Our main inspiration is the fact that we have not been to the doctor's office in years. Our health is deteriorating because of this, and even if we went now, we would die of respiratory problems because they would drown us in forms. Because we want to live, we created software for doctors to help them stop drowning us and start lifting us up.

What it does

This software is a fully encrypted dashboard for doctors. Using their credentials, they can:

  • Log in and view their patients
  • View past visits
  • Fill out new visits

But adding another form for the visit is BORING, so we made an automatic transcription tool that:

  • Listens in to the doctor's conversations with the patient
  • Automatically fills out the necessary data
  • After the chat is complete, creates a data file that can either be:
    • Downloaded
    • Saved to the database to be accessed later in the past visits page

How we built it

Component Technology Used
Backend Supabase (data structure, storage, user authentication)
Frontend HTML, CSS, JavaScript (no frameworks) — basic project structure for pages/files
Transcription Tool DeepGram API — real-time transcriptions, separates voices (doctor speaks first)
Diagnosis & Summary Gemini API with fine-tuned prompt — generates SOAP notes and suggestions

Challenges we ran into

We had a lot of trouble pushing and pulling commits from GitHub. While working on the project:

  • Sometimes we made changes to files that others were editing simultaneously
  • This caused merge conflicts when pulling
  • There were other random issues too
  • We spent significant time resolving these

Accomplishments that we're proud of

We're proud that we were able to create a complete application that doctors can use to save time with patient visits. It has actual functionality, and with a little more work, we can get it to real doctors.

What we learned

  • How to use Supabase for authentication and storage
  • How to push and pull properly through GitHub
  • The importance of working on separate files at a time to avoid merging conflicts
  • How to utilize APIs to push real-time data to our site

What's next for UF Health SmartScribe

  • Talk to doctors to fine-tune the software to fit their needs accurately
  • Fine-tune the transcription AI and Gemini AI to:
    • More accurately make transcripts
    • Format data better
    • (Possibly create our own model tuned specifically to our needs)
  • Add more features, like:
    • A patient portal for seamless connection between doctors and patients

Built With

Share this project:

Updates