Dr. Mem

This is the Omi device we are using for listening and making the conversations to memories.

Inspiration

Healthcare has a paradox: the most important moments happen in conversation, but the system captures them poorly. Clinicians are overloaded, patients forget instructions, and critical details disappear between the exam room and the chart. We wanted to build something that helps both sides of care, not another chatbot, but a "memory layer for medicine" that makes visits more accurate, more human, and more actionable.

What it does

Dr. Mem is a "native SwiftUI iOS app" paired with the "Omi wearable". There are 3 models, They are:

Education Mode: captures teaching rounds and feedback, auto-extracts “clinical pearls” and follow-up learning tasks.
Brain Dump Mode: rapid personal capture that turns quick thoughts into structured tasks and searchable memories.
Patient Encounter Mode (core): with explicit consent + clinician review, it generates Clinician Draft (SOAP/H&P-style structure + plan + action items) and Patient AVS (After Visit Summary) in plain language: next steps, medication changes, red flags, follow-up timeline.

Encounters (the killer feature)

Every patient visit becomes a structured "Encounter timeline" so clinicians can instantly review past interactions and outputs, without digging through notes.

Chat that’s actually useful

A Claude-style chat uses "retrieval from your own memories" and shows citations, so answers are grounded in what was truly captured.

How we built it

We built Dr. Mem as a native iOS app in SwiftUI with a premium “Liquid Glass” design system inspired by the Claude iOS UI. It connects to the real Omi wearable using the Omi Swift SDK, enabling live transcription directly inside the app. All data is stored on-device using SwiftData, organized into Sessions, Encounters, Memories, Tasks, Journal entries, and Chat threads. For intelligence, we use an OpenRouter LLM to handle summarization, structured extraction (Memories and Tasks generated as strict JSON), clinician-ready draft creation, patient-friendly after-visit summary generation, and grounded chat that includes citations back to the source content. The Patient Encounter workflow is designed to be simple and safe: a consent gate comes first, then record, generate, review and approve, and finally export using clipboard export for clinician documentation and PDF/share output for patients.

Challenges we ran into

Key challenges we tackled included designing a real-time capture experience that feels safe and unmistakable using a persistent “Recording ON” state, a visible timer, and a frictionless stop flow. We also focused on structuring messy, natural speech by turning raw transcripts into reliable outputs like a SOAP-style clinician draft, a patient-friendly AVS, and actionable tasks. Another major challenge was navigation: keeping the app clean and intuitive while supporting multiple surfaces like Chat, Memories, Listening, Journal, Tasks, and Encounters without overwhelming the user. Finally, we prioritized trust and safety by building clinician-in-the-loop review, adding hooks for redaction, and implementing retention controls so the product can realistically fit healthcare workflows.

Accomplishments that we're proud of

Dr. Mem stands out by offering a premium, fully native iOS experience that feels polished and consumer-ready not like an early demo. It pairs with the Omi device for real connectivity and live transcription, woven into a smooth end-to-end workflow. At the center of the product is an Encounter timeline that keeps clinician drafts and patient-facing summaries side by side, in one place, for easy review and reference. The encounter pipeline is built around precision and oversight everything is generated to be reviewed, edited, and approved, rather than running on “AI autopilot.” On top of that, Dr. Mem includes a retrieval-grounded chat interface that makes past visits and notes instantly searchable and usable, with responses anchored to the original source content.

What we learned

The real value isn’t the transcript itself; it’s converting everyday clinical conversations into structured, review-ready care outputs clinicians can actually use. Patients don’t need more data, they need clear, plain-language guidance and exact next steps they can follow. Clinicians only adopt tools that are fast, fit naturally into workflow, and feel obviously safe by design. In healthcare, UX is not a “nice to have.” If it doesn’t feel premium, simple, and effortless, it won’t get used consistently.

What's next for Dr. Mem

Build a PHI-safe Patient Encounter pipeline with transcript-first retention, no external routing by default, and an option for on-device processing or BAA-ready deployment. Add EHR-friendly export formats (SOAP drafts, discharge/AVS templates) designed for quick copy/paste into real clinical workflows. Expand into team + training features like educator review, resident feedback loops, and lightweight learning dashboards. Improve recall with stronger retrieval, including optional embeddings and encounter-aware RAG so “what happened last time?” is fast and dependable. Run a pilot with real clinicians to quantify impact: time saved, patient understanding, and follow-up adherence.