MindTrace

The stress your child can't tell you about — Mindtrace reads it live, on-device, starting in a game and headed for every screen they use.

Comment

Inspiration

Every parental-control tool on the market measures the wrong thing. Screen Time and Family Link count minutes and app categories — a proxy for what's happening to a child, not the thing itself. Meanwhile, the U.S. Surgeon General's 2023 advisory on youth mental health, Harvard's research on toxic stress reshaping the developing brain, and the WHO's own recognition of Gaming Disorder in ICD-11 all point at the same gap: nobody is measuring the body's actual stress response while a kid is locked into an engineered attention loop — infinite scroll, loot mechanics, streaks — running variable-reward psychology on a nervous system that's still under construction.

We kept coming back to one question: could a browser, with nothing but a webcam, read that signal directly — privately, without a wearable, without a single frame of video ever leaving the device?

What it does

MindTrace is "Catch the Pets," a simple falling-object game for kids ages 3–12, wrapped around a real physiological sensing pipeline:

  1. Before playing, a 20-second calibration reads the child's resting heart rate and HRV to build their own personal baseline — not a generic population cutoff.
  2. While they play, the camera continuously estimates heart-rate variability from subtle, invisible color shifts in their skin.
  3. A stress classifier compares each moment against that child's baseline, and flags persistent deviations — not single noisy blips — as "stress bursts."
  4. After the round, a parent (gated behind a simple math check, not the child) gets a report: a stress-level timeline chart, the longest sustained stress burst, a comparison against general pediatric reference ranges, and a few gentle, non-alarmist recommendations — with a one-click mailto: summary they can send themselves.

Every ounce of this runs client-side. Nothing is uploaded, ever.

How we built it

Camera → pulse signal (rPPG). We implemented the classical POS algorithm (Wang et al., 2017) from scratch in TypeScript. For a windowed, per-channel-normalized RGB signal $C_n = [R_n, G_n, B_n]^\top$, POS projects onto a plane orthogonal to the skin-tone axis:

$$ S_1 = G_n - B_n, \qquad S_2 = -2R_n + G_n + B_n, \qquad h = S_1 + \alpha S_2, \qquad \alpha = \frac{\sigma(S_1)}{\sigma(S_2)} $$

overlap-added across sliding windows into a continuous pulse waveform, then bandpass-filtered before peak-picking.

A second, heavier engine. We also converted a PhysNet 3D-CNN checkpoint from Hugging Face to ONNX and ran it client-side via ONNX Runtime Web — verified bit-identical to the original PyTorch output (max diff $1.8\times10^{-7}$) before wiring it in as an optional, more-accurate mode.

Pulse → stress. From beat-to-beat intervals we compute HRV, primarily RMSSD:

$$ \text{RMSSD} = \sqrt{\frac{1}{N-1}\sum_{i=1}^{N-1}\left(\text{IBI}_{i+1} - \text{IBI}_i\right)^2} $$

— which the psychophysiology literature (Thayer & Lane's neurovisceral integration model) treats as a marker of autonomic self-regulation.

Stack: Next.js 16 (App Router, Turbopack), TypeScript, Tailwind CSS 4, a hand-ported POS engine, PhysNet-via-ONNX Runtime Web (Worker-proxied), custom HRV/baseline/deviation logic, and Playwright for automated end-to-end verification of every pipeline stage.

Challenges we ran into

  • A calibration bug that failed 100% of the time. Our AI engine needs 20 seconds just to fill its input buffer, plus real compute time for a forward pass — but our calibration screen had a hard 20-second deadline. Every first attempt was mathematically guaranteed to fail, regardless of lighting or face angle. The fix: stop racing a fixed clock and poll for an actual result instead, with a generous ceiling.
  • A frozen game. Running the CNN on single-threaded WASM blocked the main thread for the whole inference — including the game's own canvas. ort.env.wasm.proxy = true moved inference into a Web Worker so gameplay never stutters.
  • Our own test tooling lied to us. A misplaced argument in page.waitForFunction(fn, options) silently capped every "65-second" wait at Playwright's 30-second default, making a real bug look intermittent for longer than it should have.
  • Resisting the easy fake data. It would have been simple to invent an "average child's stress score" for comparison. That dataset doesn't exist, so we used real, clearly-caveated pediatric resting-heart-rate reference ranges instead, and said plainly where our own thresholds are heuristic, not clinical.

Accomplishments that we're proud of

  • A genuine two-engine rPPG pipeline — one classical, one deep-learning — both running entirely in the browser, with the AI model's ONNX conversion numerically verified against the original PyTorch weights before we trusted a single number out of it.
  • A privacy story that's actually true, not marketing: we can point to the code and show that no frame of camera video is ever written to disk, sent over the network, or stored anywhere.
  • Catching and fixing a subtle, guaranteed-reproducible timing bug (the calibration race) through disciplined debugging rather than pattern-matching a quick patch.
  • Holding the line on honesty in a space that tempts exaggeration — a kids'-health product that says "prototype," "not a medical device," and "here's exactly where our accuracy numbers come from" out loud, on every report screen.

What we learned

That the hard part of "read a child's stress from a webcam" isn't the signal processing — it's the discipline around it. Getting the DFT-based HR estimate right,

$$ \text{HR} = 60 \cdot \arg\max_{f \in [0.7,,3.0],\text{Hz}} \left| \sum_{n=0}^{N-1} x[n], e^{-i 2\pi f n / f_s} \right|^2, $$

was the easy part once we had ground truth to check it against. The harder work was deciding what not to claim, and building every "convenient" shortcut (a plausible-looking fake stat, a race-condition-hiding retry) out of the system before it shipped.

What's next for MindTrace

  • A background layer, not a game — the same on-device sensing running quietly behind whatever a child is actually doing: Roblox, YouTube, anything — with one parent-controlled toggle and no dashboard for the child to see.
  • Dopamine-burst detection, not just stress — flagging prolonged, low-variability "locked-in" stretches typical of short-form video and loot-based game loops, alongside acute stress spikes.
  • Multi-child, multi-device profiles, so a family's baseline data travels with the child, not the browser tab.
  • Real validation — testing our accuracy claims against a reference pulse oximeter, and inviting pediatric researchers to stress-test our thresholds before we'd ever call them more than heuristic.

Built With

Share this project:

Updates

Submission history