MediScribe

What inspired this project?

Dylan's mom works for a Cosmetic Surgeon who mentioned that a virtual assistant that could transcribe a consultation would be very useful for them

How does it work?

Takes an audio recording, transcribes it to a string, passes the transcript to Gemini AI where it's converted into a report.

What work did you do?

We installed OpenAi's whisper AI dependencies on our devices to transcribe an audio file. We make an API call to Google's Gemini to parse the transcription for relevant information and generate a report. Lastly we created a React web server as the UI for the application.

Some challenges you encountered?

API calls from the React web server to the backend was challenging to debug. Working with new Languages and APIs.

Some achievements you're proud of?

Getting complex applications like Whisper and Gemini API to work. Having a nicely formatted web page that can record audio.

Some things you learned?

How to use various tools and languages to try to solve a complex problem.

What's next for MediScribe,

if we wanted to expand on this we would interview doctors to refine the report to a more appropriate format. Ensure all PIPEDA and HIPPA compliance policies are followed and more.