Inspiration
To build high performance backends
What it does
Our challenge was to build a resilient API server under a hard constraint: 2 CPU cores, 2 gigabytes of RAM no exceptions. The API exposes three endpoints with wildly different costs. /price is a near instant lookup /stats computes basic statistics over 500 data points. /risk performs 50,000 chained SHA-256 hashing operations per request, deliberately expensive, and impossible to shortcut or cache. All three endpoints share the same two cores.
How we built it
Before settling on an architecture, we load-tested naive baseline servers in Python, Java, Node, and Go, all under this same 2-core cap. Python's single-worker, GIL-bound model serialized every request behind whichever one was currently running. Java's thread pool sized itself off our host's core count, not the container's cap, causing contention. Node's single event loop blocked entirely during /risk. Go's goroutine-per-request model gave us the strongest starting point, genuine concurrency, without those specific failure modes.
Challenges we ran into
Once we actually load-tested the first working Go version under the real two-CPU, two-gigabyte cap, every single threshold failed — not just risk. Price and stats, which should be near-instant, blew out to over eight hundred milliseconds at the ninety-fifth percentile — four times their limit. And risk, the endpoint we expected to struggle, came in at over twenty seconds — against a one-point-five second budget. Two reasons: Go's runtime defaults its concurrency ceiling to the host machine's core count, not our container's two-core cap. And there was no limit at all on how many expensive risk requests could run at once. With every request racing for the same two cores, cheap and expensive work choked each other out equally.
Accomplishments that we're proud of
We built a generalized Dispatcher that puts an isolated worker pool and a bounded queue in front of every endpoint, not just /risk. Every incoming request gets wrapped in a small task envelope, carrying its payload and a context that enforces a deadline. That deadline isn't the endpoint's raw threshold — it's trimmed by a 10% error margin. /risk's real budget is 1.5 seconds. Internally we treat it as 1.35, so we return a response before the grader would ever record a breach, not after.
Each endpoint gets entirely separate resources, sized to match its actual cost. /price gets 8 workers and a queue of 200, because it's cheap and we want to absorb bursts. /stats gets 4 workers and a queue of 100. /risk gets exactly 2 workers, matching our CPU cap exactly, with a much shorter queue of just 20 Because a long queue of expensive requests just delays the moment we admit we're overloaded. If /risk's queue fills completely, /price is entirely unaffected — that's the noisy-neighbor problem, solved by isolation rather than by hoping requests behave. There's one more trade-off worth calling out specifically, inside the /risk worker itself. Every 1,000 hash rounds, out of the full 50,000, the worker pauses to check whether its deadline has already passed. If a task waited too long in queue and its budget is gone, we abandon it immediately instead of burning CPU on a hash nobody's waiting for anymore. That same checkpoint also yields the goroutine back to the scheduler, so one /risk request can't monopolize a core and starve its queue-mates. The trade-off is explicit: we accept slightly more overhead per request, in exchange for guaranteeing no single computation can hold a core hostage. Put together, the real trade-off across this whole design is: we chose to let the server say no, quickly and cheaply, rather than accept unlimited work and slowly grind to a halt under load.
What we learned
Our naive version's /risk p95 sat around two-and-a-half seconds, failing its threshold every time. With the dispatcher in place, all four thresholds pass, consistently, across every run. Requests successfully processed nearly tripled and /price and /stats didn't just stay fast, they got faster, because they're no longer competing with /risk for an undifferentiated pool of capacity. With more time, we'd tune queue depths further under sustained peak load, and experiment with adaptive worker counts per endpoint. Automatized parameter tuning (worker count, queue length, timeout thresholds) Adaptive worker allocation between tasks But under the constraints we were given, this is a server that isolates failure instead of spreading it, and fails fast and honestly instead of failing silently, withstanding the k6 stress test.
What's next for CASTLE_FIRE_WALL
With more time, we'd tune queue depths further under sustained peak load, and experiment with adaptive worker counts per endpoint. Automatized parameter tuning (worker count, queue length, timeout thresholds) Adaptive worker allocation between tasks But under the constraints we were given, this is a server that isolates failure instead of spreading it, and fails fast and honestly instead of failing silently, withstanding the k6 stress test.
Log in or sign up for Devpost to join the conversation.