ATOM: 12× more AI agent throughput per Arm core

Most optimize the model. We optimized the wire: a binary, NEON-aligned format for MCP tool calls. ~12× more agent-turns per Neoverse core, 44% fewer bytes, confirmed on Arm Performix counters.

Comment

Track: Cloud AI · Repository · Apache-2.0

Agentic AI systems repeat one message shape millions of times: an LLM runtime asks a tool executor to run a named tool with fixed arguments and gets a result back. Today that hot path runs over JSON-RPC 2.0 (MCP tools/call), which is human-readable and needlessly expensive to parse at agent scale.

ATOM is a binary, schema-aware, NEON-aligned wire format that replaces only that repeated hot path. Session setup stays plain JSON-RPC, negotiated per SPEC §7.2 with transparent fallback. Measured on Neoverse, the Arm64 silicon that cloud agents actually run on:

  • ~10.6× more agent-turns per core than simdjson (the SIMD baseline) and ~12.1× than yyjson (2.40M turns/sec/core), with ~44% fewer bytes per turn. One Arm64 instance absorbs an order of magnitude more tool-call rate before serialization overtakes the model as the bottleneck.
  • Request decode ~4.0× faster than simdjson and ~4.9× than yyjson (37.2 ns/call); columnar response decode ~9.1× vs both (6.6 ns/row); nested Tier-2 responses ~7.3× vs yyjson.
  • Confirmed independently with Arm Performix hardware-counter sampling (284,637 samples, code_hotspots recipe): 3.0% of sampled cycles in ATOM's entire decode path vs 44.9% in yyjson's (SPEC §10.2). That profile was taken on the v0.4 wire, which does strictly more work per field than the shipped wire, so it is a conservative bound.

The format itself is protocol-independent. ATOM's core (SPEC.md Part I) is positional binary frames whose layouts both ends derive at runtime from a JSON Schema, with no IDL and no codegen. It applies to any boundary with a repeated message shape and a schema-agreement moment before the repetition starts (SPEC §2.1).

MCP is the first, reference profile of that core (SPEC Part II), and the one every number here is measured on. A second profile has since shipped: OpenAI-compatible function calling (profiles/openai/PROFILE.md), held to the standing bar that a profile ships only with its own negotiation demo and measured baselines. It proves the §2.1 contract against a host that has no JSON-RPC floor to fall back on. Agent-to-agent protocols are the named candidate after that (SPEC §7.3).

The submission carries a sub-project: reusable Arm CI tooling. ATOM's headline claim is a hardware-counter measurement, so making it reproducible meant getting Arm Performix to run inside CI. Performix profiles a remote target over SSH and expects a human at a GUI, and an ephemeral CI runner is neither. Closing that gap produced two reusable pieces, shipped at tools/performix/: a composite GitHub Action that registers an Arm64 runner as its own Performix target over loopback SSH, with no external target, no secrets and no license key, and a wizard that reads recipe schemas live from the apx CLI and validates the generated profiling step through Arm's own recipe ready and validate-parameters before emitting it. It also encodes the low-sample-count fix we hit the hard way: short workloads yield too few hardware samples, so loop count and sampling frequency are first-class inputs. Anyone benchmarking on Arm can lift it wholesale, with or without ATOM.

Why it should win

The format was co-designed to the Arm microarchitecture. Its structure was sized to the silicon at the byte layout:

  • The header is 128 bits because that is exactly one NEON lane, ingested with a single vld1q_u8 instead of several scalar reads.
  • Every prefix and value is 8-byte aligned because that is what a free zero-copy int64[]/float64[] view over wire bytes requires.
  • Schema fingerprints run on runtime-dispatched hardware CRC32 (__crc32d, 8 bytes/step), with a portable fallback where the extension is absent.

Most of the speedup is structural: no field names on the wire, length-prefixed fields, no parser state machine, with the Arm-specific work as the deliberate cycle-shave on top. The structure is itself the Arm decision, made once, at design time.

It is validated on Arm at every layer: Neoverse CI captures on every push, a differential test proving the NEON path bit-identical to the scalar reference, and Performix PMU-counter attribution of where the JSON baselines burn their cycles.

Benchmarks run on real Neoverse silicon in CI on every push and upload their output as public artifacts. The baselines are chosen to be hard: simdjson (the canonical SIMD JSON parser, NEON backend, granted its padded copy and parser reuse outside the timed loop) and yyjson (one of the fastest general JSON parsers). Both directions of the agent↔tool boundary are covered, and ATOM is smaller and faster at the same time. The capacity figure is the Arm cloud metric, so ~2.4M agent-turns/sec/core is what a Graviton/Axion-class fleet gets back per core before adding instances.

What it does / functionality and output

  • libatom — a ~1,700-line dependency-free C11 library: encoder, decoder, JSON-Schema-driven schema derivation with CRC-32 fingerprints, and a Tier-2 compile-once/execute-per-response decode plan for nested results. Scalar reference paths are normative; NEON paths are verified bit-identical by a differential test.
  • SPEC.md (1.0-rc, wire revision 0x06) — a complete normative wire spec, frozen. Every open wire question is resolved, so 1.0-final will be a ratification rather than a revision. Includes the capability + fingerprint negotiation that lets ATOM slot in front of any MCP server and fall back to JSON-RPC transparently.
  • conformance/ — a 25-case golden-frame corpus, CI drift-checked against the reference encoder. Valid frames carry their decoded values and invalid frames carry their expected error code, so independent implementations have to agree on why a frame fails.
  • atom-rs/ — a second, independent implementation: a zero-dependency Rust decoder and encoder written from the spec alone with the corpus as referee. The decoder passes the full corpus; the encoder rebuilds every valid case byte-identically.
  • profiles/openai/ — the second profile: OpenAI-compatible function calling, specified, demonstrated with a runtime and executor over endpoint-level negotiation, and measured.
  • atom_mcp — an installable Python package that gateways a stock FastMCP server onto ATOM framing, plus demo bindings in Python, Node, Go, Rust, Java and C++ over one FFI surface.
  • tools/performix/ — the sub-project: a composite GitHub Action and recipe wizard that make Arm Performix profiling reproducible inside CI.
  • Benchmarks and fuzzers — the four-benchmark suite that produces every number here, seeded libFuzzer targets under ASan on Arm64, and a 302-check test suite.

Setup instructions (Arm64)

Requirements: a C11 compiler and make. Works on any aarch64 Linux or macOS machine: AWS Graviton, GCP Axion, Ampere, a GitHub ubuntu-24.04-arm runner, or an Apple Silicon Mac. x86-64 also works, via the bit-identical scalar fallback.

git clone https://github.com/Aristide021/ATOM.git && cd ATOM

make test    # 302 checks, incl. the NEON-vs-scalar differential test
make bench   # request, response, nested and agent-turn throughput
             # vs simdjson and yyjson — prints the table below
make run     # decode the canonical MCP worked example (SPEC §6)
make conformance   # regenerate and verify the 25-case golden corpus

NEON and the CRC32 extension are runtime-detected (src/atom_cpu.c); there are no configuration flags.

End-to-end against a real MCP server (needs pip install 'mcp>=1.10' numpy cffi):

make mcp-sdk-demo   # stock FastMCP server + ATOM gateway, ~30-line integration
make mcp-lib-demo   # the same path via the atom_mcp library, 5 lines
make openai-demo    # the second profile

To validate on third-party hardware instead of your own, fork the repo and push: the bench-neoverse CI job runs the full suite on GitHub's free public ubuntu-24.04-arm runners and uploads the capture as an artifact of your run. For hardware counters, dispatch the arm-diag workflow, which installs Arm Performix on that runner, profiles atom_bench and uploads the run as a .zip you can open in the Performix GUI.

Developer experience: adoption cost, measured

We measured integration cost the same way we measured latency.

  • Your existing MCP servers: 0 lines changed. The gateway fronts any stock stdio MCP server, demonstrated against an unmodified official-SDK FastMCP server in CI, and transcodes at the first hop.
  • Your agent runtime: 5 lines with the pip install-able atom_mcp, and the demo runs exactly that block in CI. Or ~30 lines by hand if you would rather own the code than take the dependency.
  • Adoption is per-tool and per-call. Underivable schemas, fingerprint mismatches, or any single call can stay on JSON-RPC on the same stream with no renegotiation. A server that has never heard of ATOM sees a normal JSON-RPC client, so trying ATOM cannot make an existing workflow worse.

Benchmarks

All numbers reproduce with make bench. Neoverse figures come from the CI artifact for the frozen 1.0-rc wire (run 30170008821).

Path Workload Neoverse vs simdjson vs yyjson Size vs JSON
Request create_event, 5 scalar args 37.2 ns/call ~4.0× ~4.9× 39% smaller
Response 100 rows × 4 fields 656 ns (6.6 ns/row) ~9.1× ~9.1× 34% smaller
Nested (Tier 2) 2 objects + 50-row table 354 ns — ~7.3× 25% smaller
Agent turn request + 100-row response 2.40M/sec/core ~10.6× ~12.1× 44% fewer bytes/turn

cJSON is in the suite too, at ~42–90×, but beating a slow DOM parser proves little and it is left out of the table.

simdjson's NEON parsing helps most exactly where ATOM's margin is smallest, the 211-byte request, and barely moves the response path, where a DOM or On-Demand walk per row cannot compete with columnar pointer-jump extraction. Even the best SIMD parser still transports field names and re-discovers structure ATOM agreed on once at tools/list.

Before and after, including the part that got worse. The v0.5 "aligned-wide" revision removed the 64 KB field limit and made every value 8-byte aligned, giving zero-copy int64[]/float64[] views directly into wire bytes. The small-frame request path retreated from ~6.7× to ~4.9× on Neoverse, since wider prefixes cost most in relative terms on a 128-byte frame, while turn-level capacity improved from 10.8× to 12.1×. We publish both directions of that trade.

A negative result, measured properly

To claw back the request-path regression we hypothesised a NEON fast path: under v0.5 alignment, adjacent fixed-scalar arguments form exact 16-byte cells, so two can be decoded from one 32-byte load. We built it twice, as a true-SIMD vld2q_u64 deinterleave and as a scalar-paired variant, verified both bit-identical to the reference decoder, and A/B benchmarked it on two Arm microarchitectures via CI (PR #1).

Each column is measured against its own machine's baseline, so the numbers are the effect of the change and not a comparison between the two cores:

Request decode, vs that core's own v0.5 scalar loop Neoverse (server) Apple Silicon (wide OoO)
NEON vld2q_u64 pair +4.5% −23%
Scalar-paired loads +9.6% −12%

The same optimization pays on one Arm core and costs on the other. Neoverse gains ~10% because the pairing removes real loop overhead; Apple Silicon's wider out-of-order core already hides that overhead and only pays the added instruction cost. At the turn level even the Neoverse gain was ~1%, too little to justify shipping two decode paths and a microarchitecture switch, so main keeps the simple loop and the experiment stays public as the record. We optimize against measurements, including the ones that say no.

Verify any of it yourself

  • Neoverse numbers on your own hardware: fork and push, or make bench on any Graviton/Axion/Cobalt instance.
  • Decoder correctness: make test — 302 checks including the differential test that pins the NEON header path bit-identical to the scalar reference over 100k random inputs.
  • The wire itself: make conformance — 25 golden frames, also downloadable as a checksummed tarball from the v1.0-rc2 release.
  • That the spec suffices to reimplement: cd atom-rs && cargo test.
  • Schema-fingerprint agreement: make bindings-all — six language bindings derive the same schema_id (0xa2febb37) from the same live JSON Schema.
  • Where the cycles go: SPEC §10.2 documents the Performix methodology step by step, and the arm-diag workflow regenerates the hotspot data on a free runner.

Challenge period

ATOM was created during the challenge period. The first commit is 3 July 2026 and the most recent is 14 August 2026, 68 commits in total, with no pre-existing codebase. The full history is public in the repository, including the CI runs and benchmark artifacts behind every number above.

Built With

Share this project:

Updates