Inspiration
Did you know physicians spend around 35% of their time documenting patient data. This weekend’s challenge was about solving this crucial bottleneck.
The idea is simple. A doctor installs the app on their desktop or laptop, opens a patient record, and starts speaking. The app transcribes the conversation then it is automatically transformed into clean, structured medical notes — ready to store, search, and upload to national healthcare systems like EESZT.
But our real innovation isn’t just technical. It’s philosophical. We provide watertight security for patient records by bringing the model to the data — not the data to the model. No data leaves the doctor’s office.
Most existing solutions send sensitive medical conversations to cloud platforms such as Microsoft Azure for speech recognition and processing. That means patient data leaves the doctor’s office and travels to remote servers.
We believe that’s the wrong direction. Even with best security practices, this introduces risks of leaking confidential patient data.
What it does
We created our app with privacy in mind. No data leaves the doctor's computer. Our desktop app (Yes, this is a desktop application in 2026, because real clinics rely on stable workstations. We designed for actual clinical workflows, not consumer tech trends.) processes microphone data locally in overlapping chunks, stitches the segments using some heuristics based on the Lehvenstein distance and then processes the notes locally via Ollama.
How we built it
Tech stack:
- Java 25 & JavaFX
- Whisper for local text transcription (https://huggingface.co/ggerganov/whisper.cpp/blob/main/ggml-small-q8_0.bin)
- Sliding window audio recording to resolve chunk boundary issues
- Lehvenstein distance-based deduplication for overlapping transcriptions
- Ollama for SOAP medical note extraction (https://ollama.com/ using
qwen3:1.7b) - SQLite for local storage
Challenges we ran into
Our app is fully cross platform, but this involved several challenges around local LLM inference. We ended up using Ollama but experimented a lot with ONNX too.
Accomplishments that we're proud of
Fully local solution, no reliance on internet connection. No patient data leaves the device. Our app is a working demo. It's not just a mock.
What we learned
Lots of things about local audio, speech and text processing
What's next for our project
More customization options, including selecting specialized local models for medical data.
Log in or sign up for Devpost to join the conversation.