50000 SHADES OF SHA

Inspiration

The name is a bad pun on the 50,000 iterative SHA-256 rounds each /risk request demands.

The engineering came from high-frequency trading platforms, where a real-time quote must never queue behind a heavy background risk calculation. That is exactly the failure the starter code exhibits: a /price lookup — a single map hit that does no computation — takes 654 ms, blowing a 200 ms bar by 3.3×, purely because it is sitting behind a 50,000-round hash loop.

The kernel idea came from Intel's multi-buffer cryptography work: a single hash chain is latency-bound, since every round waits on the one before it, but concurrent /risk requests are independent chains. Run several of them lane-parallel and a latency-bound instruction becomes a throughput-bound one.

We set out to prove that a small, dependency-free system could saturate a 2-CPU cap almost entirely with useful work — no database, no sidecar, no framework.

What it does

A price-analytics API serving /health, /price, /stats and /risk, plus durable price updates, under a hard 2 CPU / 2 GB cap.

16.6× the starter's throughput — work_score 3,540,468 against 213,874, measured back to back on one machine at one commit. 1,414,634 requests, 0.00% errors, 0 requests shed.
Keeps the 50,000-round SHA-256 workload from ever blocking a lightweight lookup: /price p95 is 1.5 ms against a 200 ms bar, where the starter needs 654 ms and fails.
Puts 93.7% of the entire CPU budget inside the hash kernel — the transport costs 28.8 µs of CPU per request.
Every threshold cleared with the tightest one — /risk — at 21% of its bar.
Survives SIGKILL and full container recreation with no database — an fsynced append-only log, replayed at boot.

How we built it

A single native binary. server.js execs it and blocks; everything that answers a request is C++.

A measured kernel ladder, not a build flag. SHA-NI multi-buffer at six lane widths, a 16-lane AVX-512 kernel, an 8-lane AVX2 kernel, and a portable scalar reference — all compiled unconditionally, each gated on its own CPUID bits. At boot the engine times every tier the CPU can actually run and keeps the cheapest per request. One binary, correct on any x86-64.
A correctness gate that runs before the stopwatch. Every tier is verified against the reference chain before calibration may select it; a tier that disagrees is disqualified and never serves. A wrong digest scores zero, so a fast wrong kernel is worth less than a slow right one.
An epoll transport with no interpreter on it — requests parsed in place, response heads that are one constant plus an integer, hash results returned over an eventfd.
A bounded hash pool sized from /sys/fs/cgroup/cpu.max — never from the visible core count, which reports the host.
A build that makes no network request at all. One g++ call against headers already in the image.

Challenges we ran into

Head-of-line blocking. One shared queue meant all three tiers landed within 5% of each other at ~650 ms — everything inherited the same wait. The starter only passes its /risk bar because the rest of it is equally slow.
Every "how many CPUs" API lies inside a container. Under --cpus=2 on a 16-thread box, nproc and sched_getaffinity both answer 16. Sizing a pool off that oversubscribes the quota 8×.
Little's law caps what scheduling can do. The load test is a closed loop, so fixing head-of-line blocking doesn't reduce concurrency — it moves it, and ~184 of 200 VUs end up parked in the risk queue. Mean /risk latency is then pinned at parked VUs / throughput regardless of queue discipline. Qualifying and scoring turned out to be the same problem.
Going native put V8's arithmetic at risk. The data series uses Math.sin, and glibc's differs from V8's in the last ulp — enough to make /stats drift. We generate the series with V8 at build time and format every double by ECMAScript's rules rather than printf.
The hybrid kernel we were sure would work. SHA-NI leaves the vector ALUs idle and AVX2 leaves the SHA unit idle, so running both in one thread should win. It measured 30–70× worse — register spills plus the AVX/SSE dirty-upper penalty. We kept it in the tree as a measured tier the calibration reports and never picks.

Accomplishments that we're proud of

3,540,468 work_score against the starter's 213,874 — 16.6× — with 0.00% errors and 0 requests shed across 1.41 million operations.
/price p95: 654 ms → 1.5 ms. /stats the same. /risk p95 at 313 ms against a 1,500 ms bar. The starter fails /price and /stats outright; we clear all four.
93.7% of the CPU budget reaching the hash kernel, with 1.98 of 2 cores in use.
The model predicted the result. Little's law said /risk p95 should land at 347 ms from measured throughput; it came in at 313 ms — 11% apart. 2.5 × requests predicted the score to within 0.11%.
We declined the exploit. Shedding /risk to farm the cheaper tiers is worth ~+5% score for 90% of the error budget. We modelled it and left it out; the shedder that exists is budgeted at half the ceiling and shed nothing across four sieges.
Byte-identical responses, verified against an oracle computed in Node from the reference implementation — not "close enough".

What we learned

Parallelising independent chains across SIMD lanes is what converts a serial, latency-bound workload into a throughput-bound one. The instruction was never the problem; running one chain through it was.
Profile before rewriting. We only removed the framework, and later the runtime, after measuring what fraction of the budget they actually cost. The transport is now 6.3% — there is nothing left there worth taking.
Queue scheduling reshapes the latency distribution but cannot manufacture throughput. LIFO moves the tail; it does not move the mean.
Run-to-run spread can exceed the effect you're measuring. An identical binary spanned 29% across four runs on a thermally-loaded laptop, and 5.3% on a machine with headroom. Any comparison worth trusting has to be back-to-back on one box — which is why the 16.6× above is, and why we don't quote a number for what trimming the build bought.
A tier you cannot execute is a tier you must be able to disqualify at boot.

What's next for 50000 SHADES OF SHA

Validate the AVX-512 kernel on real hardware. It is written, unit-tested at the instruction level, and gated — but neither development machine has AVX-512, so it has never actually run. It exists for Skylake-SP/Cascade Lake Xeons, where SHA-NI is absent and it should be the winning tier.
An SSE/SSSE3 4-lane tier, so pre-AVX2 hardware degrades a step rather than a cliff.
Feed real-time cgroup throttling pressure into the shed decision, instead of inferring load from queue wait alone.
Adapt the append-only log into a replicated state machine for multi-node deployment.