Cardio Glasses

Everyday glasses that watch high-risk hearts and speak up before an emergency.

Comment

CardioGlasses

Inspiration

All of our grandparents have lived with heart disease. We've watched the same pattern play out in each family: a doctor says stay active, walking keeps your heart healthy, and then one day a short walk ends with a racing heart that won't settle. Nobody is around, and there's no way to know whether it's normal or an emergency. After a scare like that, people walk less. The thing keeping them healthy becomes the thing they're afraid of, and their families start worrying every time a phone call goes unanswered.

Heart patients are monitored closely in the clinic and almost not at all at home, which is where things actually change. Wearables show a number, but a number isn't an answer. Is this unusual for me, given what I'm doing and what's in my chart? Should I act?

So we built a device that answers that question, in a form older adults already wear every day: their glasses.

What it does

CardioGlasses puts a pulse sensor and a motion sensor behind the ear, on the glasses someone already wears. It watches their heart all day without them thinking about it, and stays quiet until something actually matters.

  • It knows the wearer's normal. The system learns a personal resting baseline and pulls the wearer's medical record from FinchNode: their conditions, medications, and clinic heart-rate readings. Each one changes how the system watches. AFib switches beat detection to irregular-rhythm mode. A beta-blocker changes what "recovered" should look like. A high risk tier means it speaks up sooner.
  • It knows what they're doing. The motion sensor separates "high heart rate because he's walking" from "high heart rate while sitting still," and separates real beats from motion artifacts.
  • It watches what matters after activity. Most wearables only tell you your heart rate is high. CardioGlasses checks whether the heart comes back down the way it should.
  • It talks like a person. When recovery stalls, a calm ElevenLabs voice checks in: "Your heart rate is taking a while to settle. Please sit down and check your phone." The phone dashboard shows exactly why: resting, 27 bpm above usual, not coming down.
  • The wearer stays in control, with a safety net. They tap "I'm OK" or "I need help." Asking for help, or not answering within 60 seconds, sends their caregiver a text through Photon that includes the actual numbers.

It's guidance and a check-in, never a diagnosis.

How we built it

Hardware (hw/firmware)

  • ESP32 DevKit V1 with a MAX30102 optical PPG sensor and an MPU6050 IMU on a shared 400 kHz I2C bus, soldered to perfboard and mounted on the temple tip at the ear crease. The IMU sits right beside the PPG sensor so both measure the same movement.
  • MAX30102 settings: Red + IR mode, LED drive 0x7F (about 25 mA), 411 µs pulse width (18-bit), 400 Hz internal sampling with 4× averaging, giving 100 Hz output. MPU6050 set to ±4 g and ±500 deg/s with a ~44 Hz digital low-pass filter, configured through raw register writes.
  • Each PPG sample read from the sensor FIFO triggers one IMU read. Samples carry a counter rather than a timestamp, so the sensor's own clock keeps the 10 ms spacing exact.
  • Packed binary records over BLE (NimBLE): 24 bytes each (<III6h: index, IR, red, 3-axis accel, 3-axis gyro), four per notification, about 2.4 kB/s.
  • Both sensors sleep until a client connects and sleep again when it disconnects. The ESP32 re-advertises on its own, so the bridge can reconnect without a reset.
  • Runs untethered from a USB power bank, drawing about 120–160 mA.

Bridge (hw/bridge)

  • Python with bleak. It decodes the records, detects dropped samples from gaps in the index and truncated packets from size mismatches, and converts units to g and deg/s.
  • Assigns Unix-ms timestamps as wall-clock time at the first packet plus index × 10 ms, so bursty BLE delivery never distorts beat intervals.
  • Emits Contract A JSON live and saves every session for replay.

AI pipeline (ai/)

  • Produces a Reading every 2 s from a 10 s window. It includes:
    • heart rate and beat intervals from a bandpassed (0.7–3 Hz), peak-detected PPG
    • an honest 0–1 signal-quality score
    • resting/moving classification from the IMU
    • a resting baseline that freezes during episodes, so an episode is never learned as normal
    • post-exertion recovery tracking
  • A rules gate assigns normal, monitor, notify, or escalate.
  • Gemini writes the plain-English explanation. When an alert is about to fire on a borderline pulse, it also adds a signal check. Templates take over if the LLM is slow or unavailable, so an alert is never blocked waiting on it.
  • ai/clinical.py turns the FinchNode record into monitoring parameters: risk tier sets alert thresholds, AFib switches rhythm mode, rate-control medication picks the recovery comparison group, and clinic heart rate sets the starting baseline.

Backend and app (backend/, web/)

  • FastAPI with a websocket push to a phone dashboard.
  • FinchNode patient pull, falling back to a cached copy and then to local JSON.
  • ElevenLabs voice, with pre-generated fallback clips for when Wi-Fi drops.
  • A 60-second check-in timer, Photon iMessage to the caregiver, and coached exercise sessions with heart-rate zones from the patient's record.

Tooling

  • Replay mode runs real recordings through the identical pipeline.
  • A demo simulator scripts each scenario.
  • A serial and stream checker validates Contract A.
  • A comparison page runs a fixed smartwatch rule and CardioGlasses on the same 16 minutes of data.

Challenges we ran into

  • Behind the ear, the pulse is faint. A fingertip pulse is about 1–3% of the optical signal. Behind the ear we measured about 0.3–0.9%. Our first mastoid recordings were pure noise, and the detector still output a believable heart rate just by finding peaks in the noise. We moved the sensor off the bone into the ear crease, roughly tripled the LED current, and learned to judge the raw waveform by eye before trusting any number.
  • More light caused brownouts. Raising the LED current made the supply rail sag during LED pulses. The sensor silently reset to its default (off) state while BLE stayed connected, streaming zero samples. A 10–100 µF capacitor at the sensor fixed it.
  • Movement overwhelms the signal. Head motion shifts the sensor and swings the reading by thousands of counts, compared with a few hundred for the pulse. It also falls in the same 0.5–4 Hz band as heart rate, so no filter can remove it. The IMU gating exists for exactly this.
  • Contact didn't recover after movement. On the breadboard, the pulse came back 3–4× weaker after any movement. The fix was mechanical: a foam pad that keeps the sensor under constant pressure.
  • Detector errors. Peak detection double-counted the small secondary bump in each pulse, inflating heart rate by about 15%. A minimum beat spacing, a prominence threshold, and a narrower band fixed it.
  • Spectral HR must be gated. Spectral heart rate computed over a window containing motion read 127 bpm against a true value around 70, because head-shaking energy dominated the spectrum. It's only reliable on short windows the IMU marks as still.
  • Settling time. The signal needs 20–30 s after the glasses go on before it stabilizes, so baseline calibration has to wait.
  • Fit. Our glasses didn't fit our heads well, and the sensor needs steady skin contact. We taped the build on so everything stayed reversible.

Accomplishments that we're proud of

  • Validated against Apple AirPods. Worn on the glasses, the motion-gated average heart rate was 64.3 bpm against the AirPods' 66 bpm, a 1.7 bpm difference.
  • Internally consistent. Two independent heart-rate methods on the same behind-the-ear signal (beat counting and spectral) agree with a concordance coefficient of 0.87 and a bias of −0.3 bpm, with limits of agreement of −3.4 to +2.9 bpm.
  • Clean wireless data. 100 Hz over BLE, untethered on a power bank, with zero dropped samples across a 60-second validation run.
  • The motion sensor works. It automatically flagged 19% of a real recording as movement, and those flags lined up exactly with the corrupted pulse segments.
  • The medical record actually changes the monitoring. It isn't a profile page: AFib, medication, and risk tier each change how the system watches the wearer.
  • The whole loop works end to end: sensor, bridge, AI, decision, voice, check-in, caregiver text, and back to normal monitoring.

What we learned

  • With wearables, the physics comes first. Placement, contact pressure, LED current, and power integrity mattered more than any algorithm.
  • A plausible number is not evidence. We learned to judge the raw waveform before trusting a heart rate, and to report signal quality honestly.
  • Context makes alerts meaningful. The same 95 bpm is fine on a walk and concerning at rest. The same irregular rhythm is noise for one patient and normal for another with AFib.
  • Correlation isn't agreement. Comparing two heart-rate methods properly took Bland-Altman analysis and the concordance coefficient, not R².
  • Building in binary packets, frozen contracts, and replay mode early let three people work in parallel without waiting on each other.

What's next for CardioGlasses

  • Disappear into the frame. A custom flex PCB, a small LiPo battery, and a fitted mount that holds the sensor steady in the ear crease, so the glasses look like ordinary glasses.
  • Hear it from the glasses. A bone-conduction or open-ear speaker in the temple tip, so check-ins don't need AirPods.
  • Read more from every heartbeat. The PPG waveform carries more than heart rate. Recent research ties waveform shape to cardiovascular risk and even to warning signs hours before stroke. With proper clinical data, the same sensor could flag much more.
  • Prove it clinically. Validation against ECG with older adults, with heart failure and AFib patients, in real homes, followed by a clinical pilot with a cardiac rehab program.
  • Close the loop with care teams. Trends and episodes shared with clinicians, so the time between appointments stops being a blind spot.

The goal stays the same: Robert keeps walking, and his daughter stops worrying.

Built With

Share this project:

Updates

Submission history