Meridian

Project Schema

Inspiration

Doctors lose a huge part of their day to software that was never designed for how they work. Patient records are scattered across separate systems for labs, notes, medications, imaging, and scheduling, so one patient's story has to be pieced together by hand. Legacy interfaces take many clicks to reach basic information and rarely work on a phone at the bedside. Clinicians type notes long after the visit instead of talking to the patient, and a lab report that arrives on paper still has to be retyped into the chart. Most older systems store data but never analyze it, so spotting a trend across visits means exporting a spreadsheet. Screening schedules go stale: a family history that should move a colonoscopy from 45 to 40 gets noticed only if someone remembers to look. Front-desk staff face their own version of the problem, digging through insurer portals and policy documents to answer simple questions about coverage and prior authorization. We wanted to build a system where the data works for the people who rely on it, and where the AI never takes control away from the doctor. Most EMRs are not using friendly. they were not made by doctors. According to Applied Clinical Informatics (which is an official peer-reviewed eJournal), the most important. Brown PJ, Marquard JL, et al. “What Do Physicians Read (and Ignore) in Electronic Progress Notes?” Applied Clinical Informatics, 2014;5(2):430–444. DOI: 10.4338/ACI-2014-01-RA-0003 (PubMed ID 25024759).

What it does

Our platform turns raw patient data into something a clinician can understand in seconds. It summarizes a patient's history and shows an interactive 3D model of the body, with each condition highlighted where it appears. The doctor can switch specialty views to emphasize the regions that matter and slide through a timeline to see how things changed. It also charts vitals, lab results, and trends automatically, so patterns are visible without any manual analysis.

Doctors can speak or type to the system. It works out what they want, whether that is updating the chart, answering a clinical question, or checking whether screenings are current. A doctor can photograph a paper lab report, and the system reads the values off the page and turns them into proposed chart entries. Nothing the AI produces changes the record on its own. Every proposed change goes into an approval queue with a before-and-after view and its sources, and only the doctor's approval writes to the chart. A dedicated reminder assistant rechecks screening timing after every approved change, and the doctor approves any change it proposes. Due reminders go out to patients as messages that contain no health details, and a patient can opt out by replying STOP.

Answers to clinical questions are checked against trusted medical guidelines, research literature, and reputable public medical sources, and the doctor can click through to every source. A secretary view gives front-desk staff an appointment book and a read-only assistant that answers insurance questions using the patient's coverage, insurer documents, and public insurer information. Doctors and secretaries share the same insurance eligibility check.

How we built it

We started by planning the architecture and deciding how each feature would work on a phone and on a larger screen, since clinicians are rarely at a desk. We identified our main users early so every design decision had a person behind it. Doctors need fast visual overviews and a clear way to explain conditions to patients. Primary care physicians see a wide range of patients and need trend tracking and screening reminders across visits. Medical researchers need to ask questions of large datasets and get evidence-backed answers without writing custom queries each time. Front-desk secretaries, added later in the design, need scheduling and insurance answers without ever touching clinical decisions.

The backend is a Python API that connects a web dashboard with 3D rendering and a mobile app with a mic and camera. Patient data lives in a time-series-friendly Postgres database that also stores note embeddings for search, and a cloud data warehouse hosts the guidelines, research abstracts, and insurer documents. Gemini models handle request classification, the agents, lab-photo reading, and plain-English summaries. Our agents reach patient data through a custom MCP server, and each tool is scoped narrowly. The only write tool an agent has can add a suggestion to the queue. Voice sits behind a single speech hook so the mic and typed chat share one path. Lab photos are stored in a private cloud bucket. Reminders use a messaging API, and a seed script loads simulated patients, mock insurance coverage, and sample appointments for the demo. Evidence filtering happens in code instead of in the prompt, and a second model pass checks every sentence of an answer against the passage it cites.

Challenges we ran into

Our biggest challenge was planning a backend that links every feature together. Each component worked alone, but making them cooperate was far harder, and it forced us to rewrite our design several times. We started with a toggle that split voice input into dictating and asking, then realized the split caused confusion and merged everything into a single entry point that classifies each request. We started with an MCP server for our data warehouse and switched to direct REST calls to keep things simpler. We dropped an outside insurance service in favor of seeded mock coverage so the demo would be reliable. The reminder assistant caused a real design conflict, because it needed to run after every approval but could not be allowed to suggest undoing a doctor's manual edit, or to loop forever. We solved that by running it only after approvals and letting it stop once it finds nothing more to change.

Testing was the other long fight. Tests that passed in isolation failed once the features were connected, because of mismatched data formats, timing between services, and edge cases in the sample data. Failures in one service turned out to come from decisions made in another. A full test run took a long time, which slowed every fix. The evidence guardrails added their own failures: resolving redirect links, dropping sentences that only an untrusted source supported, and making sure the second check did not remove valid claims.

Accomplishments that we're proud of

We are proud of finalizing the full integration of our voice layer, our database framework, and our cloud data warehouse into one working system, and of getting a doctor's request to travel the whole path from a spoken sentence to a reviewed, sourced, approved change in the chart. We built an interactive 3D body view with specialty overlays and a timeline, plus a lab-photo reader that turns paper into structured suggestions. We built a safety model that holds up: AI can only propose, a human always approves, and every approval is logged. Our evidence rules are enforced in code, with trusted-source filtering, sentence-level verification, and a clear refusal to guess when the evidence is thin. The agents are scoped tightly, and only de-identified descriptions ever leave the system.

We also designed for more than one kind of user. Doctors, researchers, and secretaries each get a view built for their work, with shared insurance checks between them. We turned a long series of design conflicts and failing tests into a coherent, tested system and a clear, written source of truth that the whole team could build against.

What we learned

We learned how to build agents on the Model Context Protocol, with narrowly scoped tools and a custom server in front of the patient database. We learned how to use Gemini for classification, function calling, vision extraction, grounded web search, and embeddings. We learned how to run semantic search over guidelines and documents in a cloud data warehouse through its REST interface, how to use a time-series Postgres database with vector embeddings, how to design a voice-driven workflow, how to read lab photos with a vision model, and how to build a 3D body view in the browser. We also learned how to send compliant messaging reminders with opt-out handling.

The bigger lessons were about design. Integration is the hard part, and planning the data model early saves enormous pain later. Safety has to be enforced by the architecture rather than the prompt. Keeping a short list of sources of truth stops the team from building against outdated drafts.

What's next for F train was delayed

The next step is making the platform model-agnostic by integrating AWS, so hospitals and researchers can choose the AI model that fits their needs, budget, and compliance requirements instead of being locked into one provider. Our production design already points that way, with managed agent runtimes, model guardrails, a standards-based health record store, and private networking, all on AWS. We also plan to finish the live voice integration, turn on scheduled reminder sending, and load the full guideline, research, and insurer document collections into the data warehouse. Beyond that, we want to meet healthcare privacy requirements such as HIPAA with strong access controls and audit trails, connect to existing electronic health record systems, expand the 3D view to more conditions, and put the product in front of real clinicians to refine it.