Inspiration

Last month I got really sick, and I logged every symptom in a Claude Code session. It sort of worked, but it had two problems:

  • I had to pull out my phone and type every time I felt bad, and everything lived on one laptop.
  • Worse, none of it was in a shape my doctor could read. They wanted onset, location, severity and context, in the format their software already uses.

Then in the visit, the doctor typed while I talked, and an hour later I'd forgotten half of what they said.

What it does

Vexl is a symptom log you talk to.

  • Say it to your band. "Tight chest on the stairs, maybe a six." Whisper and Gemma turn it into a structured record on your laptop. Nothing goes to the cloud.
  • Built like MyChart. Same screens and same clinical words doctors already use, plus a body figure and a severity trend.
  • Records the visit. Say "I'm with the doctor", then "the meeting is done", and Gemma drafts a SOAP note.
  • The doctor approves it. Nothing is saved to your record until the doctor edits and approves the draft.
  • Private and portable. Records are encrypted on your device. The server can't read them. Export as FHIR, IPS or a spreadsheet.

How we built it

$$ \text{band} \rightarrow \text{Whisper} \rightarrow \text{Gemma} \rightarrow \text{encrypted record} \rightarrow \text{app} $$

  • The band streams 16 kHz audio over Bluetooth, transcribed locally with faster-whisper.
  • Gemma fills a fixed schema. If you didn't say it, the field stays empty.
  • Records are sealed with AES-GCM using the device's key. The server on a VM only stores ciphertext.
  • The frontend is plain HTML, CSS and JS, with a phone layout and a desktop dashboard for the doctor.

Challenges

  • Hackathon Wi-Fi blocked our laptops from reaching each other, so we moved to a phone hotspot.
  • Whisper misheard commands. "End the visit" came back as "and the visit", so we accept those variants, but only in short commands.
  • Loud rooms buried the voice commands in background chatter.
  • Keeping the model honest. It wants to invent a diagnosis. We made it leave the Assessment blank if the doctor never gave one.

What we learned

  • Doctors don't need more data, they need it in their shape.
  • Local models are good enough: a spoken visit becomes a draft note in under 20 seconds on a laptop.
  • The doctor's approve step isn't a limitation. It's why they'd trust it.

Built With

  • gemma
  • google-deepmind
  • ollama
  • pipecat-deepgram
  • superserve
  • vexo-band
Share this project:

Updates

Submission history