Login Page
Sign Up

Inspiration

A first-aid instructor once told our class, after demoing infant choking response on a manikin: "Everyone in this room will forget this in three weeks." She was right. Choking first aid is a skill you learn once at a workshop, never rehearse again, and then need at the worst possible moment — often with your own child.

CPR has manikins, apps, and recertification courses. Choking response has a poster on the pediatrician's wall. I wanted something you could actually practice — on a manikin or doll, with your phone watching your form and correcting you in real time, the way an instructor would. And because skills decay, the app should also nag you (kindly) before you forget.

That's Second Breath: a camera-coached choking first-aid trainer aligned to the 2025 AHA Guidelines for CPR & ECC and ILCOR 2025 CoSTR, with distinct protocols for infants (0–12 months: back blows + chest thrusts), children (1–12 years), and adults (back blows + abdominal thrusts). Practice is manikin-only — never on a real person.

How I built it

Client: Flutter (clean architecture: BLoC/Cubits, GetIt DI, go_router), with Google ML Kit pose detection running fully on-device — no video ever leaves the phone.
Backend: Spring Boot 3 (Java 17), stateless. Serves versioned AHA protocol definitions per age group, consumes RevenueCat webhooks to sync subscription entitlements into Firestore, and runs a daily scheduled job that sends FCM "refresher" reminders on a 90-day skill-decay window.
Voice coaching: flutter_tts at half speech rate, so instructions land while your hands are busy.

The heart of it is a three-layer pose pipeline:

Feature extractor — smooths raw landmarks with an exponential moving average and normalizes every measurement by shoulder/torso span, making it invariant to camera distance:

$$\hat{x}t = \alpha x_t + (1-\alpha)\,\hat{x}{t-1}, \qquad \alpha = 0.65$$

Kinematic classifier — a state machine that detects individual strikes from wrist velocity (in torso-spans/second), stroke depth, and recoil. Infant mode tracks each hand independently; child/adult abdominal thrusts require clasped hands — both wrists close together and moving in unison — to count.
Phase machine — walks you through the full protocol: positioning (body in frame → hands placed → confirmed ready) → 5 back blows → observed repositioning → 5 chest/abdominal thrusts → reassess and repeat, with TTS coaching at every step.

Sessions end with a scorecard: rhythm score (fraction of inter-strike gaps inside the effective 0.8–4.0s window), sequence score (did you complete milestones organically or skip them?), and a composite of the two. A 3-minute minimum keeps sessions honest.

Challenges I ran into

ML Kit lies. It returns all 33 body landmarks even when they're off-screen — it hallucinates plausible positions. Early builds happily counted back blows from an arm that wasn't in frame. The fix was gating every landmark on its likelihood score, but tuning that gate was a trap: too strict and landmarks vanish during fast hand motion (exactly when you need them most), too loose and the ghost limbs return.

My worst performance bug was garbage collection, not the model. Inference would randomly collapse from ~10 FPS to 0.2 FPS. I spent days blaming the model before realizing the camera at 720×480 was generating ~15 MB/s of frame garbage, thrashing the GC and starving inference. Dropping to low resolution roughly tripled effective FPS — ML Kit downscales internally anyway, so the extra pixels were pure cost.

Counting reps from noisy poses is much harder than detecting poses. At 6 FPS, a back blow is 2–3 frames of signal. Meanwhile users wave their arms, adjust their grip, and settle after each strike — all of which look like strikes. The rep counter needed a small army of guards: minimum stroke depth and velocity, lateral-drift rejection (waving), a ≥55% recoil requirement, a >9 spans/s glitch reset for hallucination spikes, and a 550ms cooldown that only re-arms once wrist speed calms below $0.65$ spans/s. That last one killed a maddening bug where the natural bounce after a blow registered as a second blow.

Feedback oscillation. One dropped frame would flip the UI from "you're in position ✓" back to "get in position," and the voice coach would loop. The fix was hysteresis everywhere: landmarks coast 400ms through dropouts, confirmed states stay sticky through brief tracking loss, and duplicate feedback events are deduped before reaching the user.

You can't let users skip the boring step. Repositioning between back blows and thrusts matters medically (especially the infant flip), but users just waited out the timer. Now the phase machine requires observed repositioning motion — real arm velocity after a settling window — before advancing. Annoying to build; it's the difference between a game and training.

Frame-rate diversity on Android. Thresholds tuned at 25 FPS failed at 7 FPS. The classifier now adapts: when frame gaps exceed 140ms it switches to single-frame strike confirmation with a relaxed velocity threshold, and the smoothing factor rises toward 0.85 so stale history doesn't drown fresh motion.

What I learned

On-device ML performance is a systems problem. My worst bottleneck had nothing to do with the model.
Never trust a model's output structure — trust its confidence scores. "Returns 33 landmarks" and "found 33 landmarks" are very different claims.
For real-time UX, hysteresis beats accuracy. Users forgive slightly delayed feedback; they never forgive flickering feedback.
Thresholds need empathy. The "too slow" rhythm bound is tolerant because detection latency, tracker cooldown, and TTS pauses inflate measured gaps — punishing users for my pipeline's latency would be unfair.
Medical guidelines are a spec. Encoding the 2025 AHA protocols as versioned JSON on the backend (rather than hardcoding them in the app) means guideline updates ship without an app release.

What's next

Haptic rhythm cues, a shareable family "readiness score," richer badge progression, and — the dream — validating the camera's scoring against instructor-scored sessions to see how close it gets to a human coach.