Inspiration

A polished ECG demo can hide a weak evaluation. SignalProof is for students and research engineers who want to ask a harder question: what changes when a compact neural model replaces a spectral baseline, and where does that claim stop being useful? The product keeps wrong answers, uncertainty and source limits visible instead of turning one score into a medical promise.

What it does

Six connected workspaces run without an account or model API: Signal lab compares actual trained CNN and logistic-regression probabilities on attributed 8-second examples; Stress bench applies controlled digital perturbations; Evidence exposes the protocol and measured results; Error explorer filters all 1,800 test predictions; Notebook saves and replays local experiments with validated backups; Field guide explains architecture, sources, privacy, harms and offline use. Numeric CSV import is shape-validated and remains in the browser. The source includes a complete offline ZIP and a 16-page interview manual with 32 questions and answers.

This is research-only artifact-signature classification. It does not detect disease, assess normal rhythm, provide medical advice, or decide whether an ECG is clinically usable. Reference means no extra noise was injected, not clean, normal or safe.

How we built it

Use first-channel public MIT-BIH reference ECGs and the bw, ma and em noise recordings from PhysioNet. Resample 360 to 128 Hz, create non-overlapping 1,024-sample reference windows at 10-second strides, and construct four labels by leaving a window unchanged or adding one artifact at -6, 0, 6 or 12 dB whole-window RMS SNR. This SNR definition is not identical to the original NSTDB generator.

The baseline uses 12 spectral/morphology features, training-only standardization and L2 multinomial logistic regression. The CNN is trained from scratch: Conv1D 1-to-8 with kernel9/stride4, Conv1D 8-to-12 with kernel7/stride4, ReLU, global average pooling and a four-class head. It has 816 parameters. CPU training uses seed42, AdamW .003, batch128, 32 epochs, retaining minimum validation log loss. Separate scalar temperatures fit validation data only. Both models execute through explicit JavaScript inference, checked against Python fixtures.

Evaluation

Reference records are split BEFORE constructing examples. Train: 100/101/103/105/106/107 (3,600 examples). Validation: 108/109 (1,200). Test: 111/112/113 (1,800). All versions of a reference window remain together. Noise uses disjoint time regions with gaps: 0-880s, 910-1300s, 1330-1790s. Noise recording identities remain shared, and within-split noise samples may repeat.

Actual held-out measurements:

  • Spectral baseline: accuracy 46.61%, macro F1 0.411, log loss 1.152, 10-bin ECE 0.141, multiclass Brier 0.635.
  • 1D CNN: accuracy 50.78%, macro F1 0.485, log loss 1.176, ECE 0.109, Brier 0.614.

The CNN improves macro F1 but has WORSE log loss. Balanced four-class guessing has expected accuracy 25%. These are modest semi-synthetic results, not clinical-validation results. At the exploratory 70% confidence threshold, CNN coverage is only 6.78%; selective accuracy cannot be presented as overall accuracy or a safety guarantee. Per-class F1, confusion matrices, per-record results, SNR slices, all predictions, source offsets and package versions are included. No statistical-significance claim is made.

Nineteen core checks include seven Python/JS feature and probability parity fixtures with absolute tolerance 0.00002. UI tests cover all six routes at 320/390/1280px, numeric import failures, inference, save/reload/replay, stress controls, filters, exports and an actual offline download/reopen. A fresh public clone passed tests and rebuilt the app; a full training rerun reproduced core metrics exactly and matched all source-file SHA256 hashes.

Challenges and what we learned

Splitting windows randomly would leak source ECG morphology. Splitting only the ECGs would still reuse exact noise segments. The record-first and noise-block protocol avoids those particular shortcuts without pretending that three shared artifact recordings establish broad generalization. A tiny CNN is easy to ship offline, but it does not dominate a simple baseline on every metric. Making that disagreement inspectable was more useful than hiding it.

Who benefits, and who could be harmed

Students can study reproducible model comparison and leakage-aware experiment design. Research engineers can inspect a small transparent benchmark. Patients and clinicians could be harmed if these scores are used as a diagnostic result, normal-rhythm label or ECG rejection gate. Do not use SignalProof for diagnosis, triage, monitoring decisions or patient advice. Import only de-identified research signals; the app cannot verify consent, device calibration, sampling rate or clinical meaning. Backups contain raw signals and notes and should be handled accordingly.

Limitations

Only eleven selected reference records, three held out, and three shared noise sources are used. The 1,800 windows are correlated, not 1,800 independent patients. Labels are known additive constructions, not expert clinical quality judgments. Original reference signals may already contain noise or arrhythmias. Device-external, demographic, mixed-artifact and clinical-site validation are absent. Stress-bench mixtures have no ground-truth quality labels. The exploratory abstention threshold is not validated for deployment.

What we would do with another month

Add an independent expert-labeled signal-quality dataset with compatible terms, more subjects, device-external and noise-source-held-out tests, record-level uncertainty, simultaneous artifacts and calibration-transfer checks. Compare stronger models using new held-out data rather than repeatedly tuning against this test. Work with domain experts before considering clinical use.

Attribution and AI disclosure

MIT-BIH Arrhythmia Database v1.0.0: https://physionet.org/content/mitdb/1.0.0/ . Moody GB, Mark RG. IEEE Eng Med Biol Mag. 2001;20(3):45-50. DOI 10.1109/51.932724.

MIT-BIH Noise Stress Test Database v1.0.0: https://physionet.org/content/nstdb/1.0.0/ . Moody GB, Muldrow WE, Mark RG. Computers in Cardiology 1984;11:381-384.

Both databases and adapted excerpts retain ODC-BY 1.0 terms: https://physionet.org/content/mitdb/view-license/1.0.0/ and https://physionet.org/content/nstdb/view-license/1.0.0/ . No endorsement is implied. Code is MIT-licensed separately.

Instinct AI assisted with concept, code, training scripts, analysis, tests, UI, documentation and synthetic narration. Results come from executed training/evaluation, not generated performance claims. The author should study the manual and understand the method before presenting; no claim is made of independently writing every generated line or clinically validating the system.

Built With

Share this project:

Updates

Submission history