Inspiration
Last month I got really sick, and I logged every symptom in a Claude Code session. It sort of worked, but it had two problems:
- I had to pull out my phone and type every time I felt bad, and everything lived on one laptop.
- Worse, none of it was in a shape my doctor could read. They wanted onset, location, severity and context, in the format their software already uses.
Then in the visit, the doctor typed while I talked, and an hour later I'd forgotten half of what they said.
What it does
Vexl is a symptom log you talk to.
- Say it to your band. "Tight chest on the stairs, maybe a six." Whisper and Gemma turn it into a structured record on your laptop. Nothing goes to the cloud.
- Built like MyChart. Same screens and same clinical words doctors already use, plus a body figure and a severity trend.
- Records the visit. Say "I'm with the doctor", then "the meeting is done", and Gemma drafts a SOAP note.
- The doctor approves it. Nothing is saved to your record until the doctor edits and approves the draft.
- Private and portable. Records are encrypted on your device. The server can't read them. Export as FHIR, IPS or a spreadsheet.
How we built it
$$ \text{band} \rightarrow \text{Whisper} \rightarrow \text{Gemma} \rightarrow \text{encrypted record} \rightarrow \text{app} $$
- The band streams 16 kHz audio over Bluetooth, transcribed locally with faster-whisper.
- Gemma fills a fixed schema. If you didn't say it, the field stays empty.
- Records are sealed with AES-GCM using the device's key. The server on a VM only stores ciphertext.
- The frontend is plain HTML, CSS and JS, with a phone layout and a desktop dashboard for the doctor.
Challenges
- Hackathon Wi-Fi blocked our laptops from reaching each other, so we moved to a phone hotspot.
- Whisper misheard commands. "End the visit" came back as "and the visit", so we accept those variants, but only in short commands.
- Loud rooms buried the voice commands in background chatter.
- Keeping the model honest. It wants to invent a diagnosis. We made it leave the Assessment blank if the doctor never gave one.
What we learned
- Doctors don't need more data, they need it in their shape.
- Local models are good enough: a spoken visit becomes a draft note in under 20 seconds on a laptop.
- The doctor's approve step isn't a limitation. It's why they'd trust it.
Built With
- gemma
- google-deepmind
- ollama
- pipecat-deepgram
- superserve
- vexo-band