Medical report form

• 

  Snapshot of the second field which is where the filled form can be edited before submitting.

Inspiration

Digitized records are always easy to navigate, and I have always wondered which field I could apply my skills to in order to make this even easier. I decided to focus on the medical field because the data to be documented follows a specific format and is relatively straightforward to manage. This could also be very beneficial when it comes to training models in the future.

What it does

It takes in the patient-practitioner conversation as input, along with additional diagnosis from the doctor as an audio input, and fills out the predetermined form in a specific format which can be accessed or downloaded later.

How we built it

I built it using Gradio for the frontend, and for the backend, I used audio transcription to get the context for all the questions. Then, I used a Question Answering model from Hugging Face to find answers to those questions. The answers are stored as a list and then looped through, displayed inside a text box for necessary editing if needed, and submitted. The submitted data is stored in Supabase, which can be used to download the data as a CSV when needed.

Challenges I ran into

The main challenge I ran into was the lack of computing power, which made the entire process take longer than expected, causing time lags and becoming a huge bottleneck. To address this, I switched to Hugging Face Spaces, but this caused another issue: I couldn't use a custom frontend and was stuck with Gradio. This led to some problems, such as longer wait times between audio uploads and audio recognition due to inherent issues.

Accomplishments that I'm proud of

This was my first project with Gradio as well as Hugging Face Transformers, and it was a steep learning curve. Using huge LLMs felt like I had a superpower. I felt very grateful and thankful that a mere newbie like me was offered something that powerful. I was also able to learn about two useful libraries which cemented my stay in the World of ML and AI .

What I learned

I learned to leverage LLMs with the Transformers library and to use Spaces, which deploy projects as containers—a concept that was new to me. Additionally, I learned how to read logs in HF Spaces.

What's next for Medical report form

The next progressive step is to deploy it on one of the cloud providers. I also want to give it a new frontend and backend, preferably using Flask with HTML and CSS. The current Spaces website looks very bland, and I do enjoy incorporating some color and design and Its quite hard for me to do the same in Gradio itself.

Built With

• assemblyai
• gradio
• huggingface
• python
• spaces
• supabase