Vane: One binary, every Arm core

Arm isn't one target. A NEON kernel that flies on one chip can lose to scalar on the next. Vane picks the right kernel at runtime and can't print a number it didn't measure.

Comment

Vane: One App for Every Arm Device

Tracks: Mobile AI and Cloud AI.

This project covers both tracks deliberately, because spanning them is the whole point of it. The same file, with nothing recompiled, was tested on a 2019 Android phone, a 2023 flagship phone, a Meta Quest 2 headset and an AWS Graviton4 cloud server. A submission that worked on only one of those would not be demonstrating anything.

Track What was tested Evidence
Mobile AI Snapdragon 855 phone, Snapdragon 8 Gen 2 phone, Meta Quest 2 headset 3 devices, plus the on-device Atlas demo
Cloud AI AWS Graviton4 (c8g.large, Neoverse V2) The only machine where the advanced instruction set could run at all

Vane helps software run faster on many different Arm devices, including phones, headsets and cloud servers.

Instead of creating a different version of the software for every device, Vane uses one shared library. When the program starts, it checks the device and automatically chooses the best available method.

The Problem

Arm devices are not all the same.

A method that works well on a new phone may be slower on an older one. A method designed for a phone may not use the full power of a cloud server.

Developers usually have two choices:

  • Build a simple version that works everywhere but may not use the device's full power.
  • Create separate versions for different devices, which takes more time to build and maintain.

Vane provides a third option: one version that adapts to the device automatically.

How Vane Works

Vane checks the device when the program starts and chooses the best method for that device.

After that, the program does not need to keep checking the hardware. This helps reduce extra work and keeps performance high.

Vane also compares the faster methods with a simple reference version. This makes sure the faster versions produce the correct results.

What Makes Vane Different

  • One library for many devices: The same file can run on different Arm processors.
  • Automatic device detection: Vane chooses the best option for the device being used.
  • Built-in accuracy checks: Every faster method is checked against a trusted reference. If they disagree, the check fails and stops.
  • Honest measurements: Vane only reports numbers that it actually measured. If a device will not report something, Vane reports it as unavailable rather than guessing.
  • Long-running tests: Vane can measure performance over several minutes, not only in a short burst.
  • Small AI data format: Vane can store AI data using about six times less memory, helping it run on smaller devices.

Atlas Demo

Atlas is a demonstration built with Vane.

It searches through 25,000 word vectors stored on the device. The browser only displays the results; the actual calculations happen on the Arm device itself.

For example:

  • Searching king returns queen, prince and monarch.
  • The analogy king - man + woman returns queen among its top results.

The word data takes about 5 MB in its compressed format instead of about 30 MB normally — around 83% smaller. It is small enough to ship inside the project itself, which is only possible because of the compression.

Results

The same Vane library was tested, without rebuilding it, on four different Arm devices:

  • AWS Graviton4 cloud server
  • Snapdragon 8 Gen 2 phone (2023)
  • Meta Quest 2 headset (2020)
  • Snapdragon 855 phone (2019)

Three of those are mobile devices and one is a cloud server. Nothing was recompiled between them — the same built file was copied to each machine and made its own decision on arrival.

All tested versions produced correct results on every device.

Depending on the device and task, Vane was between about 2 and 7 times faster than the basic version. The biggest improvement was measured on the AWS Graviton4 server, where one task ran about 7 times faster.

Interestingly, the oldest phone gained the most on the main task. Its basic performance was the weakest, so there was more for the faster method to recover.

Why both tracks matter here

The mobile devices and the cloud server each proved something the other could not.

Only the cloud server had the advanced instruction set, so it was the only place the newest code path could run at all — and running it there immediately exposed a real bug that had been invisible on every phone. Meanwhile, the phones were the only devices where the demo could be shown running entirely on the hardware in someone's hand, with no network and no server involved.

A library that adapts to the machine cannot be demonstrated on one kind of machine.

What We Learned

Testing on real devices revealed several important lessons:

  • A feature supported by a processor is not always the fastest option. On the newest server chip, the more advanced instruction set was actually slightly slower than the older one, because on that processor both work on the same amount of data at a time. Vane reports this rather than hiding it.
  • Two devices with similar hardware features can still have different performance. A feature list tells you what a processor can do, never what it costs.
  • You cannot know whether a short test predicts long-running behaviour without measuring it. We measured, and on these devices performance held steady — which was not what we expected.
  • A test is only useful if it tests the real code the application uses. A test that rewrites the code it is checking proves nothing.
  • Performance numbers should always come from actual measurements. Code that looks correct and compiles cleanly can still be wrong; two real problems in this project were found only by running it on a device.

Future Plans

The next steps for Vane include:

  • Automatically testing the available methods on first run and choosing the fastest one, instead of assuming the newest is best.
  • Comparing Vane with Arm's KleidiAI library, and publishing the result either way.
  • Improving temperature measurements, which vary a lot between manufacturers.
  • Testing on Arm processors that work on more data at a time, where the advanced instruction set should show a clearer benefit.
  • Supporting newer Arm hardware features.
  • Creating a public performance leaderboard for different devices.

Notes on what is not claimed

  • Laptops were not tested. Vane includes code to detect Apple and Windows-on-Arm laptops, but that code has never been run, so no laptop results are reported.
  • Arm's KleidiAI library is not used. Comparing against it is listed as future work.
  • The advanced instruction set was tested at one data width only. The project's claim that it adapts to any width is confirmed at that one width and not yet beyond it.

Built With

Share this project:

Updates

Submission history