Hear Your Heart

A maternal cardiac health companion that helps pregnant and postpartum women understand their heart in plain English, powered by wearables and AI.

Comment

Inspiration

This morning a statistic stopped us cold: heart disease is the #1 killer of women in California, and pregnant and postpartum women are among the most at risk. $60 billion goes into maternal and newborn care annually in the US, a huge portion of it treating complications that earlier detection could have prevented.

But the crisis is not only clinical. It is communicative.

When a woman leaves her OB with a handwritten note she cannot read, a diagnosis she does not understand, and no idea what questions to ask, that gap between what the doctor said and what she heard is where outcomes diverge. We built Hear Your Heart for that gap.

What it does

Hear Your Heart is a maternal cardiac health companion for pregnant and postpartum women. It meets her where she is through three entry points.

Apple Watch ECG takes her ECG export and walks her through a cardiac rhythm review, translating clinical findings into plain language she can act on.

Doctor's Note lets her photograph her handwritten discharge paper. Our OCR pipeline reads it, extracts the diagnosis, and gives her a plain-language summary plus a script of exactly what to say at her next appointment.

Maternal Risk Profile asks eight simple questions about her pregnancy health history. Our prenatal model is trained on CDC natality-derived features and returns a risk tier (Low, Medium, High), the main contributing factors, and a recommended next step. For postpartum women, the UI collects five additional delivery outcome flags for future connection to the postnatal screening backend.

All three paths lead to the same place: an immersive 3D anatomical heart visualization that shows exactly what is happening in her specific heart. She can click through her own anatomy with pregnancy-specific plain English explanations. Her profile, medications, allergies, and latest visit notes live in the left panel. Her risk assessment is on the right. Her heart is always center screen.

How we built it

Frontend: React 19 + TypeScript + Vite. Three.js + React Three Fiber + Drei for the 3D heart. Framer Motion for animations, Tailwind CSS for layout. React Router manages dedicated route components for landing, onboarding, risk profile, ECG upload, doctor note, and the heart surface.

3D Model: A real anatomical heart from Sketchfab (HannahNewey, University of Dundee Medical Art MSc), loaded as GLB with interactive raycasting, numbered anatomical callouts, and a gentle idle orbit animation.

ML Pipeline: A prenatal cardiovascular follow-up prioritization model trained on CDC natality-derived features using scikit-learn and LightGBM. Eight inputs: maternal age, pre-pregnancy BMI, chronic hypertension, diabetes, prior adverse pregnancy history, live births, smoking, and multiple gestation. Race, ethnicity, insurance, and education are excluded as model inputs per HHS Section 1557 but retained as fairness audit columns.

Backend: FastAPI with Pydantic, Python 3.12. Live endpoints include POST /api/screening/prenatal-cvd, POST /api/agents/chat, and GET /health.

OCR and Agent Pipeline: TrOCR-style handwriting recognition feeds a constrained cardiac vocabulary matcher and ICD-10 mapper, with a LangChain agent handling follow-up chat from the doctor note panel.

Research grounding: Three peer-reviewed papers behind every design decision: VR/AR in Cardiac Surgery (Brazilian Journal of Cardiovascular Surgery, 2022), Real-time NIR Cardiac Imaging (Harvard/Brigham and Women's Hospital, 2008), and Apple Watch ECG Reliability (Cureus, 2023).

Challenges we ran into

The 3D heart model uses generic mesh names, Object_0 through Object_42, with no anatomical labels. Mapping 40+ meshes to correct anatomy meant building a custom raycasting system with numbered callout anchors, world-position tracking, and manual mesh identification through console logging.

Avoiding data leakage in the prenatal ML model required careful proxy target construction. prenatal_cvd_followup_proxy_v1 uses only future delivery outcomes, not pre-pregnancy conditions, to avoid training on fields that are definitionally part of the outcome.

Mid-hackathon we had to merge two parallel development branches that had diverged significantly, requiring a full stash, pull, and manual reapplication of UI work onto the refactored route architecture.

Making a cardiac health tool feel warm rather than clinical required deliberate choices at every layer, from the particle landing animation to the glass morphism UI to the language in every explanation.

Accomplishments

We shipped a real prenatal cardiovascular follow-up prioritization model, trained on CDC natality data, connected to a live FastAPI backend, returning calibrated risk tiers with contributing factors and recommended next steps.

We built a LangChain agent that powers the doctor note follow-up chat, so women can ask follow-up questions about their diagnosis without needing to Google medical terminology.

The fairness audit evaluates model performance across race, ethnicity, insurance status, and age group without using those variables as prediction inputs.

And the 3D heart visualization is unlike anything else in the maternal health space. Watching a woman's specific affected vessel light up after she uploads her doctor note is the moment we built everything toward.

What we learned

Pregnancy is a window to future health. Hypertension, gestational diabetes, preeclampsia: these are not just pregnancy complications. They are predictors of lifelong cardiovascular disease. The postpartum period is one of the highest-risk windows a woman will ever experience, and existing technology essentially ignores it.

We also learned that accuracy is not the most important design constraint in women's health technology. Trust is. A tool that feels clinical or cold will not get used. Warm design is not an aesthetic preference. It is a clinical decision.

What's next for Hear Your Heart

Full connection of the postpartum UI form to the backend postnatal follow-up screening route, already implemented and waiting on the frontend wire.

Full OCR and agent pipeline deployment: TrOCR-style handwriting recognition feeding a constrained cardiac vocabulary matcher and ICD-10 mapper, with a LangChain agent handling follow-up chat from the doctor note panel, ready for production integration.

Real-time wearable integration beyond ECG, including HRV trends, blood oxygen, sleep, and activity data streamed continuously through pregnancy and the postpartum window.

Multilingual support in English, Spanish, Vietnamese, and Tagalog.

A provider-facing dashboard the patient can share directly with her OB, built on the same API contracts already defined.

Integration with March of Dimes and California FLOURISH Initiative workflows to reach women most underserved by existing infrastructure.

Because every warning sign deserves to be heard.

Built With

Share this project:

Updates