Echo Edge

One dependency-free C core turns heartbeats into a readiness state — Arm64 server, WebAssembly, or Cortex-M, unchanged. 19× faster, 5× smaller, proven identical to scikit-learn.

Comment

Track: Cloud AI — optimized C inference measured on Arm64 cloud infrastructure (AWS Graviton / Neoverse-N1), with WebAssembly and Cortex-M as additional targets from the same source.

Repository: https://github.com/woadi-vector/echo-edge (MIT) Live demo: https://woadi-vector.github.io/echo-edge/

Inspiration

Echo started as the usual thing: Python, scikit-learn, a pickled estimator, and a serial bridge to a host machine that had to sit next to the sensor. It worked. It also meant that a classifier doing a few hundred nanoseconds of arithmetic required a laptop, a Python runtime, and a .pkl file pinned to one library version.

Most optimization work for Arm starts with a large model and asks how to shrink it. This project started somewhere else — the model was already tiny. The interesting question was what the rest of the pipeline cost, and how much of the deployment surface could be deleted outright.

The answer turned out to be: nearly all of it.

What it does

Echo Edge turns a stream of heartbeats into an operator readiness state.

Beat-to-beat RR intervals arrive from a chest strap. A rolling 60-second window produces eight heart-rate-variability features — mean RR, mean HR, SDNN, RMSSD, pNN50, RR slope, coefficient of variation, and window coverage. Those are expressed as deviation from that individual's own enrolled resting baseline, then classified as GREEN, AMBER, or RED.

The whole pipeline is one C translation unit with no allocator, no runtime, and no dependency beyond sqrtf. It compiles unchanged for:

Target Toolchain Role
Arm64 / Neoverse gcc -mcpu=neoverse-n1 fleet-scale scoring
WebAssembly emcc -msimd128 in-browser, on-device
Cortex-M arm-none-eabi-gcc wearable firmware

The browser client reads the strap over Web Bluetooth and runs inference in WebAssembly. On an Arm-powered phone or tablet, that executes on the device's Arm cores and no heart data leaves the tab.

How we built it

Export, don't interpret. The scikit-learn forest is flattened into a single packed node array — 8 bytes per node: float threshold, int16 right_offset, int8 feature, uint8 class. Trees are emitted in depth-first preorder so the left child is always the next node in memory and needs no index at all. Eight nodes fit in one 64-byte cache line.

Leaves store distributions, not labels. scikit-learn's predict() averages each tree's class probability distribution and takes the argmax of the mean. Majority-voting each tree's own argmax is a different estimator. Leaves index a shared probability table so the C path reproduces predict() exactly.

Prove equivalence, don't assume it. The Python trainer emits parity fixtures alongside the model header. make compare builds the same sources twice — once at -O0, once optimized — and checks both against scikit-learn's own predictions. If parity fails, the build exits non-zero and every timing below it is meaningless by construction.

Measure on the actual silicon. All Arm figures come from AWS Graviton running the same source as the x86 comparison.

Results

Measured on AWS Graviton2 (aarch64, Neoverse-N1), identical source, parity verified against scikit-learn on every run.

Arm-targeted compilation — same model, compile flags only:

-O0 -O3 -mcpu=neoverse-n1 gain
feature extraction 1351.8 ns/beat 219.0 ns/beat 6.2×
classify 1107.3 ns 361.4 ns 3.1×
end-to-end 2350.0 ns/beat 482.2 ns/beat 4.9×
throughput 425,533/s/core 2,073,776/s/core —

The same source on x86 gains 2.8× from equivalent flags. Arm gains 4.9×. More headroom from the same work — that asymmetry is what this submission is built around.

Independent profile (Arm Performix). Profiling the optimized binary with Arm's own code_hotspots recipe on the same Graviton instance:

function % of samples
echo_classify 57.2
echo_features 42.0
echo_step 0.6
main 0.2

99.99% of samples land in four functions, three of them ours. No allocator, no runtime, no library churn appears anywhere in the profile — the "no dependencies beyond sqrtf" claim is confirmed by Arm's own tooling rather than asserted. Both measurement methods also agree that classification dominates feature extraction by roughly 3:2, which cross-checks the hand-rolled harness against an independent sampling profiler.

Note that Performix's counter-based cpu_microarchitecture recipe could not run: virtualized EC2 instances expose zero PMU counters to the guest. Only the sampling-based recipe was available.

Model right-sizing. Sweeping forest size, optimized build, same instance, subject-wise 5-fold cross-validation on WESAD:

trees × depth classify footprint held-out accuracy
100 × 15 965.5 ns 308.8 KB 0.903 ± 0.035
60 × 10 493.8 ns — —
40 × 8 329.7 ns — —
25 × 6 187.7 ns 27.8 KB 0.891 ± 0.034

5.1× faster and 11.1× smaller, for 1.2 points of held-out accuracy — a difference well inside one fold standard deviation. The inherited 100×15 configuration was buying almost nothing. This single change produced a larger speedup than every layout optimization combined, and it is what makes a Cortex-M target credible at all.

Worth noting what the large model looks like without a subject-wise split: it scores 1.000 in-sample against 0.903 held out. Ten points of pure memorization, and the only thing that reveals it is grouping the split by subject.

Challenges we ran into

Two bugs that were invisible in the running application.

The exporter wrote scikit-learn's internal class index rather than the class label. With a three-class model these coincide. With a two-class corpus they do not — column 1 means the second class present, not the second class defined. The application displayed a state the model could not produce, and it looked entirely plausible on screen.

The second was subtler. The C path majority-voted each tree's argmax while scikit-learn averages leaf distributions. These agree when leaves are pure and diverge when they are not — which is to say, they diverge on exactly the borderline cases the system exists to catch.

Neither was visible in the demo. Both were caught in seconds by the parity harness. That check earned its cost twice over in a single day.

Two optimizations that lost.

Packed node layout: +4% on Arm, −60% on x86. Collapsing five parallel arrays into one struct improved locality but destroyed instruction-level parallelism. At -O3 the five-array version gave the compiler independent loads it could reorder and prefetch; the packed version serializes into a genuine pointer chase. Arm's narrower reorder window and smaller cache made locality the dominant term. x86's did not. Same change, opposite outcomes.

Tree-major batching: 0.53×. Inverting the loops to stream a batch past each resident tree should amortize model loads across operators. It did not — packing had already shrunk the forest to fit in cache, so there was no streaming cost left to amortize, and the change added scratch buffers and unpredictable branches. One optimization had made the next one pointless.

Correctness cost speed, and that was the right trade. Matching scikit-learn's probability averaging made classification roughly 1.9× slower than hard voting. The faster version was computing a different model.

What we learned

The biggest win came from asking whether the model needed to be that big. Sweeping trees × depth showed identical accuracy from 100×15 down to 25×6. The inherited configuration was carrying no information the smaller one did not. That single change produced a larger speedup than every layout optimization combined — and it is what makes a Cortex-M target credible at all.

Per-operator baselining mattered more than any model change. A global z-score asks "is this heart rate high?" A per-person baseline asks "is this high for them?" Only the second question is answerable — a resting rate of 52 and one of 78 are both normal for their owners. Under subject-wise 5-fold cross-validation, personalizing against three minutes of enrollment took held-out accuracy from 0.814 ± 0.067 to 0.891 ± 0.034 against a 0.731 majority-class baseline. The variance halving matters more than the mean: performance stopped depending on which subjects were held out, which is what generalizing to a new person actually means.

Honest validation produces uncomfortable numbers. A second corpus, SWELL-KW, scored below its own majority-class baseline under subject-wise splits. Published results on that dataset commonly report ~99% using random row splits; because consecutive rows are near-duplicate windows, those figures do not survive. The negative result is in the repository rather than omitted.

Setup Instructions

Requires a C compiler and Python 3. No accelerator, no framework, no container.

git clone https://github.com/woadi-vector/echo-edge
cd echo-edge
pip install -r train/requirements.txt

make model      # trains, emits core/echo_model.h and parity fixtures
make compare    # builds twice, prints both, verifies against scikit-learn

make compare is the validation step. The parity: N/N fixtures match sklearn line confirms the C traversal reproduces scikit-learn's predictions on held-out vectors. If it fails, the build exits non-zero and the timings below it are meaningless by construction.

On Arm64

# AWS Graviton (Neoverse N1)
make compare ARM_MCPU=neoverse-n1

# Azure Cobalt 100 (Neoverse N2) — the default
make compare

Reproducing the right-sizing sweep

for cfg in "100 15" "60 10" "40 8" "25 6"; do set -- $cfg
  python3 train/train.py --trees $1 --depth $2 >/dev/null
  make clean >/dev/null && make optimized ARM_MCPU=neoverse-n1 >/dev/null
  echo -n "trees=$1 depth=$2: "; ./bench-optimized 300000 | grep "^classify:"
done

In the browser (WebAssembly on Arm)

source /path/to/emsdk/emsdk_env.sh
./wasm/build.sh
cd docs && python3 -m http.server 8000

Open in Chrome or Edge. Run simulated feed exercises the same WASM core without hardware. On an Arm-powered phone or tablet that inference executes on the device's Arm cores and no data leaves the browser tab. Safari does not implement Web Bluetooth.

Repository layout

core/     echo.h, echo.c        portable inference core
          echo_model.h          GENERATED — packed forest + scaler constants
train/    features.py           reference extractor, twin of echo.c
          train.py              fit, export C, emit parity fixtures
data/     harmonize.py          dataset adapters to one feature contract
          baseline.py           per-operator enrollment
bench/    bench.c               latency, throughput, parity
wasm/     echo_wasm.c           flat scalar surface for JS
docs/     index.html, app.js    Web Bluetooth client (GitHub Pages)

What's next

Cortex-M cross-compilation on an Arm Fixed Virtual Platform, respiratory rate derived from the existing RR series at no sensor cost, and validation against graded physiological load — the current AMBER state is a confidence band reporting model uncertainty, not a learned class, and the repository says so.

This is not a medical device.

Provenance

Echo existed before this challenge as a Python and scikit-learn pipeline with a pickled estimator, running on a host machine beside the sensor. That prior work is the starting point, not the submission.

Everything being judged here was built during the challenge period: the portable C inference core, the scikit-learn-to-C exporter, the cross-language parity harness, the packed node layout, the model right-sizing sweep, the per-operator enrollment API, the WebAssembly target, the Web Bluetooth client, the Arm64 benchmark harness, and the Arm Performix profile. The repository history reflects that.

Built With

Share this project:

Updates