AirSOAP

The whole equipment
Raspberry Pi and Microphone
Login Page
Fill out patient information
Recording the audio with live transription
Use the AI Assistant to assign speakers
Use AI Assistant to fill the SOAP form.
Use AI Assistant to find inconsistencies and things the clinician might have missed
Once finished submit the SOAP form directly to patient medical record, and the application deletes everything related to the encounter on it

Inspiration

My mom is a doctor. For the past year I have watched her see patients all day and then sit down at night to finish the notes. Hours of SOAP notes, chart reviews, writing it all up after her clinical day was already over. Sometimes she stays up late just to close her charts.

What stuck with me is that the typing doesn't only cost doctors their evenings. It costs the patient in front of them, too. A doctor who is typing is not looking at you. I wanted to see if a doctor could just have the conversation, and have the paperwork get done anyway.

What it does

AirSOAP is an AI documentation assistant that sits in the consultation room and works completely offline. The doctor records the visit with a USB microphone and the transcript appears on screen a few seconds after each pause. When the visit ends, the doctor marks who said what and AirSOAP drafts a SOAP note from what was actually said.

It also does two things plain dictation doesn't. It flags questions that were never asked, and it suggests conditions worth considering, each one backed by a quote from the conversation that the server checks is really in the transcript. If the doctor never said a diagnosis or a plan, the note says "Not documented" instead of making one up.

Everything lands in a review screen. The AI only suggests. Nothing is submitted until the doctor edits and approves it.

How I built it

The whole thing runs on a Raspberry Pi 5 with 4 GB of RAM, with no internet connection needed. Speech recognition, the language model, the server and the web app all run on the Pi itself. I also built it so doctors can connect from their own laptops or tablets over the clinic's local network. No cloud, no API bill, and the audio is deleted as soon as it becomes text. In medicine that matters more than anywhere else: a recording that never leaves the room is easy to explain to a patient.

Speech to text: whisper.cpp, with a fast model for the live transcript and a more accurate one as a fallback.
Language model: Qwen 2.5 3B through Ollama, for the SOAP note, the clinical review and speaker suggestions. Every answer has to match a strict format, and anything unusable is retried once and then reported, never shown.
Backend: FastAPI, with a small SQLite database that holds accounts only. Transcripts and notes are never stored. They live in memory and disappear when the session ends.
Frontend: React, TypeScript and Tailwind CSS.
Security: personal accounts for doctors and administrators, hashed passwords, rate limiting, and HTTPS on the clinic's local network. The language model never sees the patient's name, date of birth or record number.

Challenges

This was my first time working with a Raspberry Pi. The hardest part was making several AI components work together on 4 GB of RAM. Whisper and the language model cannot both stay loaded, so the system makes recording, transcription and each AI task take turns and unloads whichever model is not needed.

Live transcription needed measuring, not guessing. My first approach, a sliding window that re-transcribes the latest audio, fell more than 20 seconds behind the speaker on the Pi. Cutting the audio at natural pauses and transcribing each stretch once with base.en fixed it: text appeared about 4 seconds after each pause, with 8% word errors against 6% for the larger model on a 100-second test consultation. I made these decisions with a benchmark I wrote and ran on the actual Pi.

Small models can sound sure and be wrong. A smaller 1.5B model labelled speakers by assuming they take turns, so I moved that task to the 3B model and marked every AI label as unconfirmed until the clinician accepts it. I also built safeguards for the note itself: it may only state a diagnosis or plan that the clinician said out loud, and numbers that are not in the transcript are flagged.

What I learned

Building an AI application involves much more than connecting a model to a prompt. It means designing dependable workflows, validating model output, handling failures, protecting privacy, and making sure a clinician can always review and correct what the AI produced. I also learned that local AI is a trade-off: it gives more control over data, but speed and accuracy have to be measured honestly on the real hardware.

What's next

Hardware acceleration. Adding an AI accelerator or a device with a real GPU, so larger models run faster and live transcription can use a more accurate model.
Better speaker separation. Recognising speakers by voice instead of by the words.
Fine-tuning. Adapting models to clinical documentation using de-identified, properly authorized data, with rigorous evaluation so they do not introduce unsupported information.
EHR/EMR integration for clinician-approved notes.

AirSOAP is a prototype built and tested with fictional patients only. It is not a medical device and makes no compliance claims.

My goal is not to replace doctors or automate clinical judgment. It is to reduce the time and effort spent on paperwork, while clinicians stay responsible for reviewing and approving their notes. AirSOAP started because I watched someone close to me struggle with that burden, and I hope to turn that into a practical, privacy-conscious tool.