-
-
Benchmarking output on Scaleway ARM compute instance
-
Execution of state machine in Scaleway ARM compute
Inspiration
Edge devices often need to make decisions locally: shut down a machine, trigger an alarm, request an inspection, or return to a safe operating state.
I wanted to explore a different way of using AI at the edge. Instead of putting an LLM in the runtime decision loop and requiring network access for every decision, what if AI could be used as a policy compiler?
The idea behind EdgeForge is simple: use an LLM to translate a natural-language machine policy once, then compile that policy into a small deterministic state machine that can execute locally on Arm64.
For example:
"When the machine is operational, a temperature spike or vibration spike should trigger emergency shutdown. In emergency, vibration requires inspection. A manual reset returns the machine to operational."
EdgeForge converts this requirement into a structured policy, validates it, compiles it into a compact binary artifact, and deploys it to an Arm64 edge runtime.
The LLM is therefore part of the policy generation pipeline, not the runtime safety decision.
What it does
EdgeForge converts natural-language operational requirements into deterministic, locally executable edge policies.
The pipeline is:
Natural-language requirement
↓
Qwen
↓
Structured policy JSON
↓
Policy validation
↓
Deterministic compiler
↓
Compact binary state machine
↓
Arm64 edge runtime
The compiler converts states, events, and actions into numeric IDs and stores transitions in a compact matrix. Undefined transitions are explicitly represented. The generated policy is validated before the binary artifact is produced.
At runtime, telemetry is classified into events and resolved through an O(1) state/event lookup:
Temperature / vibration
↓
Event classification
↓
Compiled FSM lookup
↓
State transition + action
For the demonstration policy, the compiled state machine is only 112 bytes.
The runtime was deployed and tested on a native Arm64 Scaleway instance running Ubuntu 24.04 LTS on an Ampere M128-30 processor.
Native Arm64 benchmark results:
- Pure FSM lookup: approximately 12–13 ns per operation, or roughly 76–80 million lookups/sec.
- Telemetry classification + FSM: approximately 15 ns per decision, or roughly 66.7 million decisions/sec.
CRC32 is used as an integrity check for telemetry processing; it is not part of the policy decision mechanism.
How we built it
The project uses Qwen to convert a natural-language requirement into a structured JSON policy containing states, events, and transitions.
A Node.js compiler then validates the generated policy and converts it into a compact binary representation.
The binary contains:
- EGRF magic identifier
- format version
- number of states
- number of events
- transition matrix containing next-state and action IDs
The runtime loads this binary directly rather than interpreting the original natural-language policy or JSON at runtime.
The edge runtime combines Node.js orchestration with an Arm64-compiled C telemetry processor. The final application is packaged as a Docker image targeting Arm64.
For native validation, the same image was transferred to and executed on a Scaleway Arm64 instance. This allowed the final benchmark to be collected on a native Arm64 environment rather than treating local emulation as an equivalent hardware measurement.
Challenges we ran into
The main challenge was making an LLM-generated policy safe to compile.
An LLM can produce syntactically valid JSON that is nevertheless semantically inconsistent: unknown states or events, invalid transitions, duplicate transitions, invalid initial states, or malformed policy structures.
EdgeForge therefore performs deterministic validation before compilation. Invalid policies are rejected rather than being allowed into the runtime.
Another challenge was maintaining a clear separation between AI and deterministic runtime behavior. The temptation is to put AI inference directly into the edge decision loop, but that introduces latency, network dependency, and non-deterministic behavior. Designing the system so that AI performs policy interpretation once and the compiled FSM handles runtime decisions was a key architectural decision.
Finally, validating the Arm64 optimization required an actual native Arm64 environment. The project was therefore deployed to Scaleway rather than relying only on local development.
Accomplishments that we're proud of
The biggest accomplishment is that the project turns a natural-language operational requirement into a tiny, deterministic Arm64 runtime artifact.
The complete path works end-to-end:
Natural language
↓
Qwen
↓
Validated policy
↓
Compiler
↓
112-byte state machine
↓
Arm64 runtime
↓
Local deterministic decision
The compiler includes self-validation and reports successful validation before producing the binary.
The runtime was successfully deployed on native Arm64 and demonstrated real policy transitions, including:
- temperature spike → emergency shutdown
- vibration spike → emergency shutdown
- emergency vibration → inspection action
- manual reset → operational state
The native Arm64 benchmark reached approximately 76–80 million pure FSM lookups/sec and 66.7 million telemetry classification + FSM decisions/sec.
We are particularly proud that the performance improvement comes primarily from removing LLM inference from the runtime path, rather than simply attempting to run a larger AI model at the edge.
What we learned
The most important lesson was that an LLM does not always need to be part of the runtime.
For policies that can be represented as states, events, transitions, and actions, an LLM can act as a translator from human intent into a machine-verifiable representation. Once that representation is validated, conventional deterministic software can take over.
This changes the optimization question from:
"How do we make AI inference faster on the edge?"
to:
"Does AI need to run during the decision at all?"
For this class of edge applications, moving AI to the policy-generation stage can produce a much smaller and more predictable runtime.
We also learned that native Arm64 benchmarking matters. Running an Arm64 container through local emulation is useful for development, but it should not be presented as equivalent to measuring the application on real Arm64 hardware.
What's next for EdgeForge
The current prototype simulates the physical device actions. The next step is connecting the same compiled policy runtime to real sensors and actuators.
Potential applications include:
- industrial motor protection
- machine safety monitoring
- environmental control systems
- robotics
- remote equipment
- agricultural and industrial IoT
A future version could also support richer policy constructs, policy versioning and rollback, signed policy artifacts, stronger static analysis, and deployment directly to resource-constrained Arm devices.
The longer-term goal is to make EdgeForge a general AI-to-edge policy compiler: humans describe operational intent naturally, AI translates that intent, deterministic tooling verifies it, and compact Arm64 runtimes execute it locally.
Log in or sign up for Devpost to join the conversation.