CareBridge

Secure, offline-first AI clinical assistant for GPs. CareBridge converts conversations into structured notes locally, eliminating admin burnout and ensuring 100% data privacy.

Comment
  • * Inspiration * * Today, the market is flooded with AI scribes and clinical assistants capable of summarizing doctor-patient conversations. However, we noticed a critical flaw: the quality and accessibility of these tools often just come down to how much money a clinic can spend on expensive cloud APIs.

While large, well-funded hospitals can afford premium, cloud-dependent solutions, small and rural practices are being left behind. In Hungary, the healthcare system is facing a real-time crisis: there are currently nearly 1,000 vacant General Practitioner (GP) practices, and the trend is worsening. The remaining doctors are severely overwhelmed, spending up to 40-50% of their day on administrative tasks rather than patient care. Furthermore, these rural clinics often struggle with unstable internet connections and lack the budget for expensive SaaS subscriptions. We wanted to build a tool that solves this exact problem: bringing state-of-the-art AI directly to the doctor's desk, completely offline.

    • What it does * * CareBridge is a secure, offline-first AI clinical assistant designed specifically for GPs. It completely automates clinical documentation without relying on the internet. Once the doctor gets explicit verbal consent from the patient (complying with the Civil Code), CareBridge records the consultation and provides a real-time, on-device transcription. When the visit ends, a single click triggers our local AI to transform that raw transcript into a structured, standardized medical document (SOAP note: Subjective, Objective, Assessment, Plan). The doctor remains in the loop to review, make quick edits, and seamlessly export the final note to their local Electronic Medical Record (EMR) system.
    • How we built it * * From day one, we committed to a 100% offline architecture to ensure maximum data privacy and zero cloud costs. Instead of relying on commercial cloud APIs like OpenAI or Azure, we built CareBridge exclusively using local, open-source Large Language Models (LLMs). We integrated an efficient, on-device speech-to-text engine alongside lightweight, quantized LLMs optimized to run smoothly on standard clinic desktop PCs. Everything—from audio processing to text generation—happens entirely on the edge.
    • Challenges we ran into * * The most significant challenge we faced was getting local, lightweight LLMs to consistently generate highly structured and accurate clinical outputs. While massive cloud models can easily follow complex formatting rules, coaxing a smaller, locally hosted model to accurately summarize a messy, real-world doctor-patient conversation into a strict SOAP note required intense optimization. We had to heavily refine our prompt engineering, context management, and output parsing to ensure the model extracted all vital clinical information without hallucinating or dropping important details.

The biggest help came from Google, specifically their MedGemma 1.5 4b model which is specifically trained for medical tasks. QLoRA finetuning with Unsloth proved complicated, as several experiments to improve its summarizing performance led to worse performance. The best example was when during finetuning it was prompted to first write down the most important keywords, but since the training data did not have these, the model was very unstable and unusable.

Getting training data itself was also hard, as there are no real human-made doctor-patient conversation datasets online, let alone ones with high quality SOAP summaries. Due to this, we used the omi-health dataset, which was generated by OpenAI's GPT4o model. The biggest imporvement to our project would definitely come from higher quality data.

    • Accomplishments that we're proud of * * We are incredibly proud to have successfully run an entire speech-to-text and NLP pipeline completely offline on consumer-grade hardware with excellent response times. We proved that you don't need expensive cloud infrastructure to achieve high-quality clinical documentation. Furthermore, we are proud that by processing everything locally, we created a system that inherently and effortlessly complies with the strictest data protection frameworks, including the GDPR and the Hungarian Health Data Act.
    • What we learned * * We gained deep technical knowledge in model quantization, edge AI deployment, and prompt engineering for small-parameter models. Beyond the code, we learned a tremendous amount about the harsh realities of primary healthcare. We realized just how heavy the administrative burden is on GPs and how critical "Privacy by Design" is when dealing with highly sensitive medical and voice data.
    • What's next for CareBridge * * Our immediate next step is to finalize direct API integrations with the most widely used Hungarian EMR systems (such as MedMax and EESZT) to make the export process truly one-click. From there, we plan to launch a pilot program involving 10 rural GP practices to gather real-world feedback, test the system across various clinic hardware setups, and further fine-tune our local models for even greater speed and medical accuracy.

Built With

Share this project:

Updates